Back to overview
Downtime

Network outage

Feb 16 at 10:08am GMT
Affected services
changelog*.fly.dev

Resolved
Feb 16 at 02:03pm GMT

The app has been serving all requests successfully for the last 15 minutes.

The total app downtime was 4h. Out of all requests served, 13.72% failed with a 503 response status.

While the app is now restored, we continue monitoring the situation while implementing the following mitigations:
1. Running the app in multiple regions
2. Having a standby platform ready to go

Updated
Feb 16 at 01:48pm GMT

The provider restored service, app is now serving requests, we continue monitoring the situation.

Updated
Feb 16 at 01:18pm GMT

We are still in the process of setting the app up on a new provider.

The existing provider is still working on a fix.

One of the mitigations is bound to get this problem resolved.

Updated
Feb 16 at 12:38pm GMT

This is now approaching 3 hours which makes it the biggest provider outage to ever affect us. While everyone upstream is hard at work, we are kicking off "lift & shift".

Updated
Feb 16 at 12:18pm GMT

The network outage is still ongoing. We are waiting on the provider.

Updated
Feb 16 at 11:48am GMT

14% of all our requests have returned 503 since this outage began. We are still waiting on a fix from our provider.

Updated
Feb 16 at 11:18am GMT

The network outage is still ongoing. We are waiting on the provider.

Updated
Feb 16 at 10:58am GMT

10% of all our requests have returned 503 since this outage began. We are still waiting on a fix from our provider.

Updated
Feb 16 at 10:38am GMT

The network outage is still ongoing. We are waiting on the provider.

Created
Feb 16 at 10:08am GMT

There is a confirmed network outage at our provider. All requests which cannot be served as stale from the CDN are expected to fail with response status 503.