POSTMORTEMS

Interrupciones

URnetwork crece rápido, y el crecimiento encuentra los límites de un sistema. Cuando damos con uno, lo contamos aquí: cuándo empezó y terminó, qué viste, qué falló y qué cambiamos.

Últimos 3 meses

Cada fila es un mes, con los días alineados. La altura y el color muestran cuánto del servicio se vio afectado.

oct 2026
sept 2026
ago 2026
  • Normal
  • Degradado
  • Interrupción parcial
  • Interrupción grave
  • Interrupción total

octubre de 2026

3 interrupciones

21–25 oct 2026

99 h 44 min

Runaway connection churn filled the database disk and took the platform down

A provider-quality filter switched itself on at its default date and emptied two selection pools, and our hosted proxy service began churning through short-lived connections faster than the database could serve them. For two days connections failed or stalled for many people while database storage grew with the churn, until on Oct 23 a logging fault filled the disk and the platform was down for 16 hours, then 6 more for the storage expansion. That evening the same churn jammed payment reservations, and for about five hours apps could find providers but not send traffic.

Empezó 21 oct, 00:00 UTCTerminó 25 oct, 03:44 UTC
  • Interrupción total23 h 32 min
  • Interrupción grave14 h 04 min
  • Interrupción parcial43 h 27 min
  • Degradado17 h 38 min
Leer el postmortem

19 oct 2026

9 h 16 min

Stalled database connection pools took providers offline

The pools that connect our servers to the database stalled and could not refill. A cleanup job then mistook the stalled servers for dead ones and removed them, and 97% of providers dropped off the network for almost three hours.

Empezó 19 oct, 09:56 UTCTerminó 19 oct, 19:12 UTC
  • Interrupción total2 h 47 min
  • Interrupción grave16 min
  • Interrupción parcial26 min
  • Degradado4 h 05 min
Leer el postmortem

10–13 oct 2026

80 h 19 min

Three days of database overload, and the failures it set off

Our main database ran at its limit for three days, and every timeout added retries and more load. While it was overloaded, four separate failures hit users: a new connection check that disconnected healthy providers, a load-balancer collapse, a key rotation that broke new sign-ins, and a stalled connection pooler.

Empezó 10 oct, 00:22 UTCTerminó 13 oct, 08:41 UTC
  • Interrupción total2 h 10 min
  • Interrupción grave35 h 35 min
  • Interrupción parcial19 h 34 min
  • Degradado23 h
Leer el postmortem