SourceHut was disrupted by a DDoS attack.
Over the holidays SourceHut experienced a DDoS attack primarily targetting git.sr.ht. For a period of several hours we experienced severe performance issues and intermittent outages as a result, primarily affecting git.sr.ht.
Our upstream network provider identified the malicious traffic and blackholed the appropriate routes, but due to a configuration error SourceHut’s AMS subnet was entirely null routed for a period of about one hour yesterday, causing a complete outage for all services. Once the configuration error was corrected, the DDoS continued to cause performance issues that primarily affected git.sr.ht, until the attack died down later yesterday afternoon.
The approximately 1-hour total outage of our AMS installation caused a severe network partition that disrupted systems hosted in other installations, notably our build workers. These issues were not understood until this morning, when the ensuing problems were corrected. However, builds submitted in the meanwhile were cancelled, and users will have to resubmit any builds from this period that still need to be run. You can do this by opening the affected build on the builds.sr.ht web UI and clicking “Resubmit”.
Additionally, there have been some issues updating our status page, which resulted in poor communication during the incident. An old outage notice was never properly closed, and during the outage updating the status page was a challenge – and a distraction. We elected to defer the solution until today to allow our engineers to enjoy what remains of their holiday weekends. The status page issues have now been resolved.
We apologize for the inconvenience.
(10:00 UTC — Apr 6)Attacks have resumed.
DDoS attacks targetting git.sr.ht have resumed, disrupting performance and availability. We are looking into mitigations.
(15:30 UTC — Apr 6)