We have completed repairs and service is online. We will continue to monitor throughout the night to ensure there are no further occurrence of the network latency.
Update 12/07/2024 21:21 PM 18 days
we've replaced the router and are observing metrics offline before restoring service. eta for service restore 4:30PM Eastern
Update 12/07/2024 20:59 PM 18 days
we are programming a replacement core router as we're still seeing minor latency. estimated replacement around 430pm
Update 12/07/2024 20:41 PM 18 days
the network is being tested with the replacement switch and we will revert to the primary switch at 4PM eastern
Update 12/07/2024 20:25 PM 18 days
we are going to reboot the core router to also clear out any issues which will interrupt service for 5-10 minutes
Update 12/07/2024 20:10 PM 18 days
the reboot has completed and the issue does appear to have been transient. We will continue to monitor and at 330PM eastern we will perform a temporary switch to the backup switch to ensure that its fully provisioned in the event that the issue crops up again.
The switch to the backup switch and back is estimated to take up to 20 minutes and connections will be temporarily broken.
We will restore connection to the primary after validation is complete
Update 12/07/2024 19:53 PM 18 days
Prior to replacing the switch at 3PM we are going perform a reboot of the current switch to ensure the issue isn't transient. If we continue to see the same performance issues we will switch to the pre-programmed standby switch
Update 12/07/2024 19:38 PM 18 days
At 3pm eastern we will be replacing the core switch for exchange. Replacement is estimated to take up to 10 minutes for service to stabilize
Update 12/07/2024 17:37 PM 18 days
We are beginning on site work and will keep this post updated for any anticipated outages that will last longer than 10 minutes
Update 12/07/2024 00:59 AM 19 days
We are working to address the password prompts in Outlook which appears to be networking hardware related. We will be performing further diagnostics and potentially emergency hardware replacement over the next 24 hours. In the event that service becomes unreachable we will provide an update to this event with an estimated time period.
Currently the issue appears to causing authentication prompts but generally do not appear again for a few hours after reauthenticating.