Build Service on Openpak Status

[Resolved] Infrastructure provider outage

Sun, 24 May 2026 02:09:00 +0000

One of Flathub’s infrastructure providers experienced an outage, which caused multiple services to become unavailable. This issue has now been resolved by the provider.

Any builds that were affected during the outage have been restarted.

If a pull request test build is still failing, maintainers can restart it by commenting bot, build on the pull request.
If an official build from a merged pull request is broken, please open an issue.

(14:27 UTC — May 24)

[Resolved] Build status updates failing

Thu, 07 May 2026 16:03:00 +0000

GitHub pull request status updates, branch commit statuses, and flathubbot comments were failing due to a missing token in the build service configuration, despite builds being completed successfully. As a result, affected pull requests did not receive status updates or completion comments, and branch commit statuses were not being updated.

The issue has been resolved now, and all affected open pull requests should be updated with their build statuses. Some commit statuses may still remain outdated, but this does not affect the publishing process.

If any pull request completed successfully but still lacks a status update, please restart the build by commenting bot, build on the pull request. (10:50 UTC — May 8)

[Resolved] Publishing delay due to delta regeneration

Wed, 25 Mar 2026 00:58:00 +0000

We are in the process of regenerating deltas for the current apps. The process takes an exclusive lock of the underlying ostree repository, preventing new builds from being published. If you have an urgent update to release, please contact us on our Matrix channel. (02:14 UTC — Mar 25)

[Resolved] Signing outage

Tue, 17 Feb 2026 13:56:00 +0000

An issue with hardware security key is currently preventing builds from being signed with our GPG key. This affects both direct uploads and builds performed on Flathub infra. Since no builds can be signed thus published, the build service including test builds are also disrupted. (13:56 UTC — Feb 17)

We are in contact with the data center staff to investigate the issue physically. (17:49 UTC — Feb 19)

Data centre staff attempted reseating the signing key across available USB ports without success. We are now arranging delivery and installation of a replacement key. (16:46 UTC — Feb 20)

We successfully managed to install the new Yubikey. (14:04 UTC — Feb 23)

Information

We will soon restart all affected official builds in batches and they should be published within a couple of hours. Affected direct uploaders will be informed to restart builds from their end. Once this process is complete, some services which were disabled (to reduce impact) will be re-enabled one by one.

Once the test build service is available, any affected test builds on pull requests will need to be manually restarted by commenting “bot, build” by the respective maintainer or contributor.

Please allow us some time to finish the entire process outlined above. Publishing may take longer than usual due the large number of builds in the queue. Please open an issue on GitHub for any further query about missing or unpublished builds. Further updates will be posted in the statuspage.

All affected builds were successfully restarted and they finished building. (16:40 UTC — Feb 23)

The repository server is still experiencing some issues. (17:30 UTC — Feb 23)

The incident should be resolved now and services are being restarted. (18:50 UTC — Feb 23)

[Resolved] CDN cache invalidation issue

Fri, 13 Feb 2026 05:38:00 +0000

Due to a brief downtime, one of the origin servers missed purging operation and was serving stale and non-existent index files. As a result Flathub builds failed while attempting to download dependencies and any general Flatpak clients experienced hangs or download failures.

The issue was resolved after the cache was invalidated. (07:00 UTC — Feb 14)

[Resolved] Build service outage

Sun, 03 Nov 2024 19:05:00 +0000

There is an ongoing outage that is causing builds to fail at the commit stage. This affects all builds done through Flathub buildbot and may affect direct uploads as well.

We are trying to resolve it. (19:05 UTC — Nov 3)

Resolved: This should be resolved now (19:39 UTC — Nov 4)

[Resolved] Blocked publishing due to ongoing repository pruning

Tue, 14 May 2024 10:00:00 +0000

Resolved: The pruning has completed, freeing almost 7TB of disk space. Affected builds have been restarted in Buildbot. (17:50 UTC — May 15)

The pruning process is still ongoing. It may take about 12 hours more. (05:30 UTC — May 14)

We are in the process of pruning the ostree repository from old builds. The process requires an exclusive lock of the repository, meaning that publishing is not going to be possible until it finishes. We will manually publish the outstanding builds afterwards. (19:00 UTC — May 14)