EXCH080 Temporary Outage
Incident Report for Manage Protect Pty Ltd
Postmortem

To Customers Affected by the Recent Service Interruption: Every day, Manage Protect dedicates itself to providing the highest levels of security to our customers and their data, as well as delivering on our 99.999% uptime service level agreements. There are very rare instances where we have to place the priority of the former over the latter. We made this very difficult decision, yesterday, when our Security team noticed unusual activity on one of our domains. As cybercriminals continue to become more sophisticated in their attempts to attack network systems, allowing activity like this to continue could have resulted in your highly sensitive and critical information being exposed, prolonged system shutdowns, and other significant damage to the hosted exchange and its customers. In order to protect your organization and data from this potential cyberthreat, we made the difficult decision to immediately block access to this domain. This domain serves approximately 2% of our customers and your account was unfortunately one of those impacted. As a result, access to your email was disabled. Our highly experienced Security, System Administration, and Technical teams worked around-the-clock to bring the domain back online. We also engaged external expert security firms to conduct an exhaustive review to help us analyze what had happened and ensure that no damage had been done. Fortunately, our extensive security controls were able to identify the potential threat, and, based on the information currently available to us, we have no evidence at this time that any customer data was exfiltrated or damaged. Service was restored earlier this morning, and you should now be able to access your email. Emails that were received while the domain was offline remained in the queue, and you should now have access to these. We have never before proactively blocked service to a full domain due to a security concern. However, for the safety and protection of our customers and their data, we felt this was the necessary approach given the circumstances. That said, we know how important it is for our customers to have the peace of mind of knowing that your communication services will be available whenever you need them. We realize how vital Manage Protect’s services are to your business, so situations like this must and will remain an anomaly. To that end, as a gesture of goodwill, we will be issuing account credits to all impacted customers, and please know that we will continue to invest significantly, utilize the best people and technologies, and work tirelessly to meet your expectations, keep you safe, and earn your trust, each and every day. We sincerely apologise for the disruption to your business.
Manage Protect

Posted May 17, 2019 - 10:40 AEST

Resolved
The access to your email has now been restored and all inbound emails have been delivered successfully to the mailboxes. We apologise for the disruption that this incident may have caused.
Please note that no loss of data occurred and that your email account is restored with all incoming emails fully queued in your inbox.
We made the difficult decision to take a subset of email accounts offline yesterday when our security team alerted us of unusual activity on one of our domains. While cybercriminals continuously advance their hacking techniques, we are also committed to protecting the security and data of our customers.
Later today we will follow-up with further details about the incident, the decisions made, and the precautions taken.
If you are still not able to access your email account, please contact support.

support@manageprotect.com
chat.manageprotect.com
1300 657 500
Posted May 17, 2019 - 08:12 AEST
Update
Our security experts have made further progress to bring your domain safely back online. We still do not have a firm ETA as to when connectivity will be restored, but again, good progress has been made. We apologize for the disruption, and want to remind you that all emails will remain in the queue and will be delivered once the domain has been restored.
Posted May 16, 2019 - 11:46 AEST
Update
We continue to make progress in restoring connectivity to the domain. At this time, we still do not have a definitive ETA as to when connectivity will be restored. We are stepping through the process as quickly as possible while continuing to prioritize the safety of the service. We will continue to provide regular updates.
Posted May 16, 2019 - 08:08 AEST
Identified
Exchange, Skype for Business and other services tied to MPexchange on exch080 (US server) are affected.
Users receive an error "Site cannot be reached" when trying to connect to their mailboxes. Users are also unable to access Skype for Business and other services. Administrators cannot manage their account services through Control Panels. We still do not have a firm ETA as to when connectivity will be restored. We apologize for the disruption.
Posted May 16, 2019 - 07:24 AEST
Update
Exchange, Skype for Business, Email Archiving, and other services are temporarily unavailable for users located on EXCH080 domain. Administrators cannot manage account services through Control Panels. All inbound emails are queued and will be delivered once services are restored.
Posted May 15, 2019 - 19:21 AEST
Investigating
We are currently investigating issues with the EXCH080 exchange server including login failures. This outtage affects only US hosted exchange accounts.
Posted May 15, 2019 - 16:23 AEST
This incident affected: MPexchange.