Incident Report for Workplace
Postmortem
On 8th August and 10th August Workplace users in the Australia/New Zealand and Canada regions experienced an issue that caused pages, including the login page, to fail rendering and were not displaying appropriately. As a result, users were not always able to log in, and navigate the product.
We have identified the root cause to be the interaction between our web applications and our Content Distribution provider, where the caching provided by our provider interfered with how the pages were returned to the user when requesting it.
On the 8th August, we disabled the caching feature which resolved the issue, however, we found on the 10th when the issue reoccurred that the caching functionality gets re-enabled automatically.
Therefore, we have temporarily removed the Content Distribution provider from our infrastructure while we work on a permanent resolution.