POSTMORTEMS

Outages

URnetwork is growing fast, and growth finds the limits of a system. When we hit one, we write it up here: when it started and ended, what you saw, what went wrong, and what we changed.

Last 3 months

Each row is a month, days aligned. Height and color both show how much of the service was affected.

Oct 2026
Sep 2026
Aug 2026
  • Healthy
  • Degraded
  • Partial outage
  • Major outage
  • Full outage

October 2026

3 outages

Oct 21 – 25, 2026

99 h 44 min

Runaway connection churn filled the database disk and took the platform down

A provider-quality filter switched itself on at its default date and emptied two selection pools, and our hosted proxy service began churning through short-lived connections faster than the database could serve them. For two days connections failed or stalled for many people while database storage grew with the churn, until on Oct 23 a logging fault filled the disk and the platform was down for 16 hours, then 6 more for the storage expansion. That evening the same churn jammed payment reservations, and for about five hours apps could find providers but not send traffic.

Started Oct 21, 00:00 UTCEnded Oct 25, 03:44 UTC
  • Full outage23 h 32 min
  • Major outage14 h 04 min
  • Partial outage43 h 27 min
  • Degraded17 h 38 min
Read the postmortem

Oct 19, 2026

9 h 16 min

Stalled database connection pools took providers offline

The pools that connect our servers to the database stalled and could not refill. A cleanup job then mistook the stalled servers for dead ones and removed them, and 97% of providers dropped off the network for almost three hours.

Started Oct 19, 09:56 UTCEnded Oct 19, 19:12 UTC
  • Full outage2 h 47 min
  • Major outage16 min
  • Partial outage26 min
  • Degraded4 h 05 min
Read the postmortem

Oct 10 – 13, 2026

80 h 19 min

Three days of database overload, and the failures it set off

Our main database ran at its limit for three days, and every timeout added retries and more load. While it was overloaded, four separate failures hit users: a new connection check that disconnected healthy providers, a load-balancer collapse, a key rotation that broke new sign-ins, and a stalled connection pooler.

Started Oct 10, 00:22 UTCEnded Oct 13, 08:41 UTC
  • Full outage2 h 10 min
  • Major outage35 h 35 min
  • Partial outage19 h 34 min
  • Degraded23 h
Read the postmortem