"After a period of extended monitoring, we’ve confirmed through customer reports and service health telemetry that the issue is resolved."
Posted Nov 26, 2024 - 13:25 PST
Monitoring
Below are the latest updates from Microsoft stating with the oldest to the latest:
11/25/2024 - 6:10PM" Our monitoring indicates that a large portion of affected users and services are seeing recovery following our mitigation efforts. We're working on addressing the lingering regions that are still seeing small impact to fully restore service availability, which we still expect to complete by Monday, November 25, 2024 at 7:00 PM PST"
11/25/2024 - 7:31PM " We’ve successfully restored functionality for all previously impacted services and users with the exception of Outlook on the web, which is showing prolonged impact for a small number of users. We’ll continue carefully monitoring the service health and focus on troubleshooting this persisting impact to fully recover for the remaining affected users. We'll provide a new timeline within the next update."
11/25/2024 - 8:32PM "Impact to core services have been restored with the exception of Outlook on the web, which we’ll continue to monitor and actively troubleshoot until full recovery.
This quick update is designed to give the latest information on this issue."
11/25/2024 - 9:04PM "We're continuing to address lingering impact to the Outlook on the web service that is still affecting some users. In parallel, we're investigating some mail queuing delays that is resulting in mail taking longer than expected to be delivered. Due to the impact of this incident, we will enter a period of extended monitoring prior to declaring this issue resolved."
11/26/2024 - 12:04AM "We’ve isolated the cause of mail queue delays and have restarted the affected infrastructure to drain stalled queues. We’ll remain in an extended monitoring phase until this draining is completed and we can consider the incident fully recovered.
This quick update is designed to give the latest information on this issue."
11/26/2024 - 1:03AM "We're still addressing the remaining impact to the Outlook on the web service that is affecting some users. We’ve applied mitigation actions to reduce the mail queues and we’re continuing the extended period of monitoring to ensure stability continues."
11/26/2024 - 2:02AM "While we continue our period of extended monitoring, the availability of the Outlook on the Web service has reached expected availability levels. We’re continuing to optimize the environment to address the remaining impact.
This quick update is designed to give the latest information on this issue."
11/26/2024 - 2:59AM " From monitoring service telemetry, most users should now experience relief. We’ve completed our optimizations and we're continuing our period of extended monitoring to ensure the availability remains stable."
11/26/2024 - 4:53AM "We’re continuing our period of monitoring service telemetry, which shows the service availability has remained healthy."
11/26/2024 - 7:58AM "We’re conducting additional optimizations to further improve service availability, which remains healthy. We're closing in on a full mitigation and we're closely monitoring telemetry to ensure this trend continues."
Posted Nov 26, 2024 - 08:14 PST
Update
Below is the latest update from Microsoft:
"The service is continuing to incrementally recover. We're forecasting that impact will be fully remediated by Monday, November 25, 2024 at 7:00 PM PST, though we're closely monitoring our progress and will provide an updated timeline should the estimate change."
Posted Nov 25, 2024 - 16:13 PST
Update
Below is the latest update from Microsoft:
"We identified a change in the environment that resulted in an influx in request retries routed through affected servers. Our optimizations, which enhanced the infrastructure's processing capabilities, continue to provide incremental relief. We're monitoring the service and continuing our work to perform any follow-up actions or opening additional workstreams needed to fully resolve the problem.
We understand the significant impact of this event to your organization, we're treating this issue with the highest priority, and we're working to provide relief as soon as possible."
Posted Nov 25, 2024 - 14:29 PST
Update
Below is the latest update from Microsoft:
"Services are showing significant signs of restoration following our infrastructure optimizations to redistribute traffic and restart affected servers, upwards of 90 percent recovery as we continue to increase processing capability. We're continuing to perform appropriate optimizations on the subset of remaining affected machines to resolve the issue for all users."
Posted Nov 25, 2024 - 13:02 PST
Update
Below is the latest update from Microsoft:
"Our mitigative actions haven't provided relief as expected, and a portion of infrastructure remains in an unhealthy state. We determined that some of the targeted server restarts did not succeed due to processing issues, which are under investigation. We’re currently focused on spreading traffic to healthy infrastructure, and we're seeing some recovery."
Posted Nov 25, 2024 - 11:22 PST
Update
Below is the latest update from Microsoft:
"We're continuing to reroute traffic to alternate infrastructure and have reinitiated targeted server restarts to ensure the fix takes effect as expected. We're monitoring to confirm the restarts proceed successfully. We don't yet have an estimated time to resolution; however, we'll provide one as soon as it becomes available"
Posted Nov 25, 2024 - 10:43 PST
Identified
Microsoft is aware of the following issues but is working to resolve them.
Here is the latest update:
Microsoft is rerouting traffic to alternate infrastructure to expedite mitigation. In parallel, they're resolving complications leading to the delays in targeted server restarts.
ISSUES:
Users may be unable to access Exchange Online using the following impacted connection methods: - Outlook on the web - Outlook desktop client
Users may be unable to use the following features within Microsoft Teams: - create or update Virtual Events, including webinars and Town Halls. - unable to access or modify their calendar in Microsoft Teams. This would include loading calendar, viewing meetings, creating/updating meetings and joining meetings. - unable to create chat, add users and create or edited meetings. - unable to create or modify new teams and channels. - may be unable to update presence. - may be unable to use the search function. - may not see updated list of files and links failing to load within the Chat shared tab.
Users may be unable to use the search feature within SharePoint Online.
Users may experience errors running flows that utilize cloud connectors in Power Automate for Desktop.
Users may be unable to access their bookings within Microsoft Bookings.
Posted Nov 25, 2024 - 09:06 PST
This incident affected: Microsoft 365 Services (Sharepoint Online, Outlook Online, Teams, Misc.).