Notice history

~0.0 ms

Feb 2025

Mar 2025

Apr 2025

~0.0 s

Feb 2025

Mar 2025

Apr 2025

Feb 2025

Mar 2025

Apr 2025

Feb 2025

Mar 2025

Apr 2025

No notices reported this month

No notices reported this month

Resolved
February 13, 2025 at 12:10 PM
Resolved
February 13, 2025 at 12:10 PM
We have switched back to our original networking, and all functionality and performance is restored. Hetzner has confirmed that is was indeed a vSwitch issue that caused the network outage.
Monitoring
February 12, 2025 at 7:07 PM
Monitoring
February 12, 2025 at 7:07 PM
We have implemented a fix involving tailscale to route around the networking issue, and functionality appears to be restored. We will continue monitoring the situation.
Update
February 12, 2025 at 4:43 PM
Update
February 12, 2025 at 4:43 PM
We have learned that the backbone connection between our cloud servers and the baremetals that host our database is nonfunctional for any new cloud servers. We are currently working with Hetzner to resolve this issue.
Investigating
February 12, 2025 at 3:31 PM
Investigating
February 12, 2025 at 3:31 PM
We are currently experiencing an issue with our DB connection pooler connecting to the database. We are currently investigating.

Resolved
February 11, 2025 at 2:59 PM
Resolved
February 11, 2025 at 2:59 PM
Performance issues have been resolved.
Investigating
February 11, 2025 at 2:42 PM
Investigating
February 11, 2025 at 2:42 PM
Node have all been restored, but the underlying cause of the issue seems to still be present (i.e. massive resource utilization across all nodes). Still investigating.
Identified
February 11, 2025 at 2:21 PM
Identified
February 11, 2025 at 2:21 PM
The nodes hosting mastodon.online for kubernetes seem to be offline. We are currently working to replace them with new nodes as quickly as possible.

Feb 2025 to Apr 2025

mastodon.online - Notice history