Dark city with warning sign showing "Email outage"

Rackspace security incident causes a hosted exchange failure

Rackspace’s managed Microsoft Exchange services have been suspended due to a “security incident,” company claimed.

The following details are taken from the company’s most recent incident report, which was time-stamped on December 3 at 01:57 Eastern Time.

On Friday, Dec 2, 2022, we became aware of an issue affecting our Hosted Exchange environment. We proactively powered down and disconnected the Hosted Exchange environment while we triaged to understand the extent and the severity of the impact. After further analysis, we have figured out that this is a security incident.

The affected services include MAPI/RPC, POP, IMAP, SMTP, ActiveSync, and the Outlook Web Access (OWA) interface, which is used to access the Hosted Exchange instance and handle email online.

The official Rackspace status website provided an update on the issue, even though the initial updates had little information besides that there was an outage and that it was being investigated.

According to the status page:

We are actively working with our support teams and predict our work may take several days. We are investigating reports of connectivity issues to our Exchange environments. Users may experience an error upon accessing the Outlook Web App (Webmail) and syncing their email client(s).

A few hours later, at 1:54 PM, Rackspace reported they were still in the “investigation phase” of the outage and were trying to figure out what went wrong. The outage had been lasting since 6:36 AM, according to the Rackspace updates, which had been describing it as “connectivity and login difficulties.”

Customers who were affected were instructed to visit the status page for the most recent updates. Even though those also missed information on the cause of the issue.

Recommendations from Rackspace

A security problem involving Rackspace’s hosted Exchange service wasn’t made public until 1:57 AM on December 3rd, over 24 hours later. It was also mentioned in the statement that the Exchange environment had been shut down and disconnected by Rackspace staff.

A “major breakdown” was described by Rackspace four hours later. As a temporary fix until they could identify the issue and get the system back up, they started giving their clients free Microsoft Exchange Plan 1 licences on Microsoft 365.

The incident report from Rackspace includes detailed instructions on how to install the free licences and transfer user mailboxes to Microsoft 365.

Twelve hours later, that afternoon, they added information to the status page, indicating that their security team and other specialists were still attempting to fix the issue.

Customers of Rackspace who were upset with the unavailability announced intent to migrate to a different, more transparent managed service provider (MSP) as well as demanded an ETA from the firm on social media.

The exact cause of the outage was revealed by Rackspace at 01:57 AM EST, over 24 hours later. A security problem that was “limited to a piece of our Hosted Exchange platform” forced the business to disconnect the Hosted Exchange environment.

Rackspace Tweet About Security Issue
Rackspace Tweet About Security Issue

Rackspace suffered significant impacts

Concerning the security alert, Rackspace has not made any disclosures. A vulnerability usually plays a role in a security event.

The effect of the vulnerabilities was explained in an advisory released in October 2022:

An authenticated remote attacker can perform SSRF attacks to escalate privileges and execute arbitrary PowerShell code on vulnerable Microsoft Exchange servers. As the attack is targeted against Microsoft Exchange Mailbox server, the attacker can potentially gain access to other resources via lateral movement into Exchange and Active Directory environments.

The most recent status report, as of December 5th, noted that the service is still down and that clients are advised to switch to the Microsoft 365 service.

The possibility of a data breach involving consumer information has not been showed. This incident is still taking place.
Help your colleagues keep a security-first mindset and boost your human firewall by starting your Phishing Tackle security awareness training today with our two-week free trial.

Recent posts