All Systems Operational
University Networks Operational
eduroam ? Operational
Pomona Campus Operational
East Valley Clinic Operational
Rancho Mirage Operational
COMP-Northwest ? Operational
Oliver Station Operational
VPN - GlobalProtect Operational
CHS-Northwest ? Operational
Academic Services Operational
Canvas Operational
Elentra ? Operational
EDU Rotations Operational
Mediasite ? Operational
Top Hat Operational
Administrative Services Operational
Banner ? Operational
Cognos Operational
Raiser's Edge Operational
bswift ? Operational
Misc. Operational
ADP Operational
Clinical Systems Operational
EPIC Operational
Axium Operational
ImproMed Operational
Campus Services Operational
Self Service (Banweb) Operational
YOU Portal Operational
Intranet Operational
www.westernu.edu Operational
Telecom ? Operational
Electricity Operational
RightFax Operational
Microsoft 365 Services Operational
Active Directory Federated Services (Single Sign-On) ? Operational
Sharepoint Online Operational
Outlook Online Operational
Teams Operational
Misc. Operational
Zoom Operational
Zoom Meetings Operational
Zoom Video Webinars Operational
Zoom Chat Operational
Zoom Cloud Recording Operational
Third-Party Services Operational
TeamDynamix (support.westernu.edu) Operational
ExamSoft Operational
Slate (apply.westernu.edu) ? Operational
Qualtrics Operational
Miscellaneous Operational
Miscellaneous Operational
Internal API Testing ? Operational
Operational
Degraded Performance
Partial Outage
Major Outage
Maintenance
Past Incidents
Mar 29, 2023
Resolved - Microsoft has resolved this issue.

Details from Microsoft:

Title: Users' file libraries weren't refreshing when navigating to another library within the Files tab in Microsoft Teams

User impact: Users' file libraries weren't refreshing when navigating to another library within the Files tab in Microsoft Teams.

More info: This issue specifically impacted scenarios in which users were attempting to access multiple file libraries via the Files tab in Microsoft Teams. While the initial attempt to access a file library succeeded, when affected users attempted to navigate to a subsequent file library, they were again presented with the initially accessed library.

While we were focused on remediation, impacted users could bypass the impact by selecting the menu marked by a down arrow to the right of the "Files" menu at the top of the Files tab, then selecting "Reload Tab" to manually reload the affected library.

Current status: The cache has cleared by itself and we've confirmed that the issue is now resolved.

Scope of impact: Impact was specific to some users who were served through the affected infrastructure.

Start time: Sunday, March 26, 2023, 11:25 PM (3/27/2023, 6:25 AM UTC)

End time: Monday, March 27, 2023, 11:26 PM (3/28/2023, 6:26 AM UTC)

Root cause: A recent change to a component of Microsoft Teams infrastructure, responsible for delivering Files tab contents, introduced a code issue that redirected requests to a cached file library, resulting in impact.

Next steps:
- We're reviewing our service update and changes procedures to prevent this issue from happening again.

This is the final update for the event.

Mar 29, 08:05 PDT
Update - Microsoft has identified the issue and root cause. They anticipate the issue will be fixed early Wednesday morning.


Details from Microsoft:

Title: Some users' file libraries aren't refreshing when navigating to another library within the Files tab in Microsoft Teams

User impact: Users' file libraries aren't refreshing when navigating to another library within the Files tab in Microsoft Teams.

More info: This issue specifically impacts scenarios in which users are attempting to access multiple file libraries via the Files tab in Microsoft Teams. While the initial attempt to access a file library succeeds, when affected users attempt to navigate to a subsequent file library, they're again presented with the initially accessed library.

While we're focused on remediation, impacted users can bypass the impact by selecting the menu marked by a down arrow to the right of the "Files" menu at the top of the Files tab, then selecting "Reload Tab" to manually reload the affected library.

Current status: We've received confirmation that the fix successfully addresses the issue, and impact is remediated after users manually clear their cache. We expect the cache will clear by itself and no further manual actions will need to be taken to experience remediation by our next communications update.

Scope of impact: Impact is specific to some users who are served through the affected infrastructure.

Start time: Monday, March 27, 2023, 2:02 PM (9:02 PM UTC)

Estimated time to resolve: We anticipate the cache will be cleared and the issue will be fully resolved by Wednesday, March 29, 2023, 4:00 AM (11:00 AM UTC)
.

Root cause: A recent change to a component of Microsoft Teams infrastructure, responsible for delivering Files tab contents, introduced a code issue that's redirecting requests to a cached file library, resulting in impact.

Mar 28, 14:46 PDT
Identified - We have received notification from a few customers that the Files tab on some Microsoft Teams channels/teams is not properly showing up. This has manifested itself in such ways where the contents from one team/channel (that you already have access to) appears in another team/channel. Customers have had some success with fully closing out Teams and re-opening and/or trying to update Teams.

Microsoft has released an advisory for a rendering issue that we believe to be related. We are keeping an eye on that advisory and will provide relevant updates when available.

Microsoft Service Health Advisory: TM533773

Mar 27, 16:04 PDT
Mar 28, 2023
Mar 27, 2023
Mar 26, 2023

No incidents reported.

Mar 25, 2023

No incidents reported.

Mar 24, 2023

No incidents reported.

Mar 23, 2023

No incidents reported.

Mar 22, 2023

No incidents reported.

Mar 21, 2023

No incidents reported.

Mar 20, 2023
Resolved - This incident has been resolved.
Mar 20, 21:30 PDT
Monitoring - IT has restarted the wireless servers for the Lebanon campus in an effort to resolve connection issues.

Users on all locations may have lost connection to eduroam wirelss network but connections should be restored.

We are monitoring the situation.

Mar 20, 13:46 PDT
Completed - The scheduled maintenance has been completed.
Mar 20, 07:00 PDT
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Mar 17, 18:01 PDT
Scheduled - Banner and all related systems, including Banner 8 & 9, Self Service Banner, YOU Portal, Banner Document Manager (BDM), eProcurement, CashNet, etc., will be unavailable.

This is due to an Oracle 19C update.

Mar 15, 14:51 PDT
Mar 19, 2023

No incidents reported.

Mar 18, 2023

No incidents reported.

Mar 17, 2023
Completed - The scheduled maintenance has been completed.
Mar 17, 20:00 PDT
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Mar 17, 18:01 PDT
Scheduled - The 3rd Friday of each month is the designated IT maintenance window. For additional information please visit: https://support.westernu.edu/TDClient/1848/Portal/KB/ArticleDet?ID=89703
Mar 15, 14:52 PDT
Completed - The scheduled maintenance has been completed.
Mar 17, 03:30 PDT
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Mar 17, 00:00 PDT
Scheduled - Announcement to Users
On Friday morning, March 17, 2023, OCHIN will be performing maintenance on Epic Production and ancillaries. Downtime is scheduled for 12:10 am to 3:30 am PT so we can complete monthly maintenance tasks and apply the Epic March Service Pack and Enhancement Pack.

Please review the ‘Who/What is affected?’ section below, which calls out the services that will be impacted.

Who/What is affected?
All users who access Epic Production and ancillary tools and applications listed below:
• Epic Production (PRD) will be offline
• Epic Shadow (SHD) will be online
• Electronic Prescribing – Superscripts will be offline
• Epic Link will be offline
• Haiku/Canto will be offline
• My Chart will be offline
• Care Everywhere will be offline
• Mirth Interface servers will be restarted and intermittently offline affecting Health Information Exchanges (HIE)
o All ADT, ORM, ORU, DFT, MDM, Immunizations, file transfers interfaces will be affected
o All queued messages will be sent once servers are back online
• OnBase server will be offline, and the following will be unavailable:
o Scanning and indexing
o Scanning and viewing documents in Front Desk Scanning, ROI Module Scanning, and Referral Module Scanning
o Viewing documents in patient charts
o Accessing OnBase Report Services
• Hillrom (formerly Welch Allyn) database server will be offline
• Viewing and saving dental images via MiPACs/Wisdom will be unavailable
• Clarity database ETL Processing Delays
o As a result of PRD maintenance, service pack and quarterly release Epic upgrade cycles, there will likely be delays in our Clarity ETL process as the process will begin later than usual, causing the delay in completion time. This affects all Clarity database reporting including our Clarity replication members, Business Objects reports, custom Clarity reports, Tableau data sources, and any Epic-Crystal reports or Clarity-based objects in Hyperspace.

Why this downtime?
This is routine scheduled maintenance

Mar 13, 10:50 PDT
Mar 16, 2023

No incidents reported.

Mar 15, 2023

No incidents reported.