6/27/2023 - Systems Outage Alert - Fiber Outage affecting multiple systems
Incident Report for Bucks County Community College
Resolved
We are considering this issue to be resolved. All of our services are operational and have been stable since late Wednesday afternoon.

Thanks,
Help Desk and Services Center
Posted Jun 29, 2023 - 11:29 EDT
Monitoring
Access to our systems appears to be restored for both remote and on campus users. The website, Self Service, old MyBucks, Report Manager and Global Protect are available from on or off campus. We're continuing to monitor the situation over the next several hours but right now, we are up and running.

Thanks for your patience,
Help Desk and Services Center
Posted Jun 28, 2023 - 17:15 EDT
Update
We are continuing to work on a fix for this issue.
Posted Jun 28, 2023 - 15:48 EDT
Update
Latest update from our fiber vendor - For the safety of the field teams, access to the location of damage will not be available until the local utility has completed repairs. The local utility (PECO) is currently on-site placing new power lines. Our line crew is working to identify slack cable in order to complete fiber repairs once access is granted. Our splicers remain on stand-by. Additional updates will be provided at 4:00 PM ET or upon the availability of new information.

Thanks
Help Desk and Services Center
Posted Jun 28, 2023 - 14:32 EDT
Update
Our fiber vendor continues to await access to the damage location from the local utility company. Crews have been sourced and once given the all clear, they will begin repairing the damaged fiber. Some staff have been able to access some applications including the website that they could not get to earlier. But this is hit or miss and most people who were affected are still unable to access the applications.

The vendor is providing us another update at 2:00. We'll update you as new information becomes available.

Thanks,
Help Desk and Services Center
Posted Jun 28, 2023 - 12:52 EDT
Update
The issues affecting access to our systems is continuing. Two of our three ISP's were affected by this outage. Some traffic is getting through and some traffic is working on campus but not off and vice versa. Here's a recap of what systems you may not be able to access.

The Doylestown PSTC is offline.

Remote Access - doesn't affect everyone seems to depend on your service provider or location
bucks.edu
Global Protect (VPN) which means you can't access Colleague UI

Remote and Campus Outages
Self Service
Old MyBucks
Report Manager
ac.bucks.edu

Campus Outages
Pearson Vue
QMinder
My Mathlab
Workday
Some Printing
Bucks Website is available but running slow especially searches

Students who just registered for Summer Three won't be loaded into Canvas until access to Report Manager is restored.

The latest report from the scene is that PECO is still working on repairing the downed power lines. The fiber repair crews and splicers are on stand-by waiting for PECO to finish.

Thanks,
Help Desk and Services Center


status.bucks.edu
Posted Jun 28, 2023 - 10:11 EDT
Update
Status remains the same. Our ISP is waiting for PECO to complete their work before they can move in and repair the damaged fiber. The break has also knocked the Doylestown PSTC completely offline. We still do not have an ETA.

Thanks,
Help Desk and Services Center
Posted Jun 28, 2023 - 05:43 EDT
Update
One of our ISP's field crews dispatched and isolated the impairment to an intersection in Southampton, Pennsylvania. The crews noted there are multiple telephone poles, power lines and fiber optic cables down. The local power company is on site starting their repair efforts. The local power company must complete their repairs and make the site safe for the field crews to conduct their work. There is no estimated time to repair. Additional updates will be provided at 12:30am, ET, or upon new information.

I have turned off notifications as it's late. Status can be checked at status.bucks.edu or the Slack channel.

Thanks,
Help Desk and Services Center
Posted Jun 27, 2023 - 23:08 EDT
Update
Global Protect is also affected by this outage which means access to Colleague Web UI is impacted.

Thanks,
Help Desk and Services Center
Posted Jun 27, 2023 - 19:53 EDT
Identified
We are experiencing intermittent issues with access to the "old" MyBucks and the Bucks website. These appear to be storm related and not everyone is affected. We are working with our Service Providers to restore access. It does not appear to have affected anyone who's on campus. We will provide additional updates as we get more information.

Thanks,
Help Desk and Services Center
Posted Jun 27, 2023 - 18:57 EDT
This incident affected: Bucks Website, MyBucks - old, Student Planning, Network, Colleague UI, Workday Production, Global Protect, and Bucks AC.