Back to overview
Downtime

Network outage

Feb 16, 2025 at 10:08am UTC

Resolved
Feb 16, 2025 at 2:03pm UTC

The app has been serving all requests successfully for the last 15 minutes.

The total app downtime was 4h. Out of all requests served, 13.72% failed with a 503 response status.

While the app is now restored, we continue monitoring the situation while implementing the following mitigations:
1. Running the app in multiple regions
2. Having a standby platform ready to go

Updated
Feb 16, 2025 at 1:48pm UTC

The provider restored service, app is now serving requests, we continue monitoring the situation.

Updated
Feb 16, 2025 at 1:18pm UTC

We are still in the process of setting the app up on a new provider.

The existing provider is still working on a fix.

One of the mitigations is bound to get this problem resolved.

Updated
Feb 16, 2025 at 12:38pm UTC

This is now approaching 3 hours which makes it the biggest provider outage to ever affect us. While everyone upstream is hard at work, we are kicking off "lift & shift".

Updated
Feb 16, 2025 at 12:18pm UTC

The network outage is still ongoing. We are waiting on the provider.

Updated
Feb 16, 2025 at 11:48am UTC

14% of all our requests have returned 503 since this outage began. We are still waiting on a fix from our provider.

Updated
Feb 16, 2025 at 11:18am UTC

The network outage is still ongoing. We are waiting on the provider.

Updated
Feb 16, 2025 at 10:58am UTC

10% of all our requests have returned 503 since this outage began. We are still waiting on a fix from our provider.

Updated
Feb 16, 2025 at 10:38am UTC

The network outage is still ongoing. We are waiting on the provider.

Created
Feb 16, 2025 at 10:08am UTC

There is a confirmed network outage at our provider. All requests which cannot be served as stale from the CDN are expected to fail with response status 503.