On 9th of June 2023 Microsoft reported on the Azure Status page, that the preliminary root cause behind an outage that impacted the Azure Portal globally was described as a traffic “spike.”
Customers attempting to access portal.azure.com reported issues connecting and seeing a warning that says:
“Our services aren’t available right now. We’re working to restore all services as soon as possible. Please check back soon.”
Other Microsoft websites were also impacted including Entra Admin center (entra.microsoft.com) and Intune (intune.microsoft.com).
Microsoft described the primary root caus of the issue with the following:
“We identified a spike in network traffic which impacted the ability to manage traffic to these sites and resulted in the issues for customers to access these sites,”
“We engaged in different workstreams applying load balancing processes in addition to the auto-recovery operations in place in order to mitigate the issue. Additionally, we are continuing to monitor the platform health.”
Attack By Anonymous Sudan
Even though Microsoft did not provide any details as to why the Azure Portal was experiencing connectivity issues, Anonymous Sudan, a known threat actor, claimed to have carried out the DDoS, Distributed Denial of Service, attack which triggered the spike in network traffic that Microsoft observed.
It is believed that the same hacktivist group has also claimed to be targeting US companies in protest against the US interference in Sudanese internal affairs. Others believe that these threat actors might be linked to Russia.
One week prior to this attack the same group claimed another DDoS attack on the Microsoft portals of Outlook and OneDrive.
The Outlook outage began Monday evening and was rectified in the early hours of Wednesday. It was disclosed by Redmond (Microsoft) that other services and features were also impacted besides Outlook. SharePoint Online and OneDrive for Business.
When Microsoft was approached by BleepingComputer for comment about Anonymous Sudan’s claims, they responded with the following quote:
“We are aware of these claims and are investigating,”
“We are taking the necessary steps to protect customers and ensure the stability of our services.”
Steps Being Taken By Microsoft to Reduce the Impact of such Incidents
Microsoft has implemented a number of mitigations either implemented or in the process of being implemented. The mitigations are as follows:
- Make the Azure portal more efficient so that it scales up quickly – Completed.
- Increase the Azure Portal scale so it can handle high demand leads more efficiently – Completed.
- Block invalid requests & server responses more aggressively – Completed.
- Use proactive logic to adjust traffic blocking and throttling rules – In Progress.
- Improving the Internal Azure Portal monitoring to detect such signs more quickly and efficiently – In Progress.
- Making the Azure Portal startup process quicker – In Progress.
What customers can do to make potentially work around such incidents
- Azure CLI and Powershell were not affected by this incident, and as a result partners and customer could continue to manage their Azure Environments using these tools.
- Azure Resource Manager (ARM) REST API’s as well as other tols using API endpoints and scripting could have been used to manage resources during the outage (https://learn.microsoft.com/en-us/rest/api/resources/).
- Evaluate the reliability of your applications using the guidance from the Azure Well-Architected Framework and the interactive Well-Architected Review (https://learn.microsoft.com/en-us/azure/well-architected/resiliency/).
- Lastly ensure that the right people in your organization get notified about any service issues through the configuration of Azure Service Health Alerts, which can in turn trigger emails, SMS, push notifications, webhooks and more (https://learn.microsoft.com/en-us/azure/service-health/alerts-activity-log-service-notifications-portal?toc=%2Fazure%2Fservice-health%2Ftoc.json).
Reference:
Leave a Reply