CIL request for Flathub

[ramcq]

If you are interested in filing a request for access to the CNCF CIL, please fill out the details below.

First Name

Robert

Last Name

McQueen

Email

rob@endlessm.com

Company/Organization

Endless Computers / Flathub

Job Title

VP of Deployment

Project Title

Flathub

Briefly describe the project

Flathub is a build and distribution service for containerized Linux desktop applications in the Flatpak format. We're looking to use CNCF infrastructure as a case study for dynamically scaling the build workers available to build/publish FOSS Linux desktop applications, and sharing technology and learnings between cloud and desktop containerization.

See https://flathub.org/ and https://flatpak.org/

Which members of the CNCF community and/or end-users would benefit from your work?

Any user of Linux on the desktop benefits from the ease of distribution, robustness and security that's brought by containerized desktop applications as offered by Flatpak. Although decentralization is key to the Flatpak tools, to allow any Linux ISV to publish apps how they wish and be successful on their own terms, the success of the Flatpak format relies on users having a central go-to place for users to find top OSS and existing Linux desktop apps. Flathub serves that need so that there will be a receptive audience for ISVs to publish their applications in the Flatpak format. As well as bringing diversity and robustness to the Linux desktop ecosystem, the adoption and development of container tech on the desktop space provides additional use cases, functionality and testing of the underlying/shared stack that also powers cloud containers.

Is the code that you’re going to run 100% open source? If so, what is the URL or URLs where it is located?

Yes. Workers for the BuildBot instances run the configuration pushed from the master here: https://github.com/flathub/buildbot-config/

Builds are run in a sandbox set up by flatpak-builder and flatpak, both at https://github.com/flatpak/

We will flesh out the Ansible playbooks at https://github.com/flathub/ansible-playbook to handle automatic set-up of workers on the CNCF bare metal hosts.

Depending which Flatpak is being built, the different manifests can be found here: https://github.com/flathub/

Any non-OSS components which are included in Flatpaks are not built/downloaded/distributed by the Flathub infrastructure, but are downloaded on the end-user system as needed.

What kind of machines and how many do you expect to use (see: https://www.packet.net/bare-metal/)?

Initially, Type 0 and Type 2A as needed during times of peak demand when existing workers are all busy. Intel architecture workers are more needed at the moment. We would likely move to larger Intel nodes as needed rather than spawn multiple nodes.

What OS and networking are you planning to use (see: https://help.packet.net/technical/infrastructure/supported-operating-systems)?

CentOS 7 is used for Flathub hosts and worker set-up will be automated with Ansible. We will set up a WireGuard VPN so that the bare metal hosts can communicate securely with the Flathub core infrastructure.

Please state your contributions to the open source community and any other relevant initiatives

I've been working and contributing to Open Source projects for nearly 20 years, including Debian, freedesktop.org, GNOME and most recently Flatpak and Flathub where I am in charge of the infrastructure. I'm one of the original founders of Collabora, a leading worldwide OSS consultancy and worked there for 10 years as the CTO. I am experienced in contributing to and guiding OSS projects in solid technical directions which also successfully deliver the benefits of FOSS to commercial adopters and end-users.

How will this testing advance cloud native computing (specifically containerization, orchestration, microservices or some combination).

Containerizing desktop applications allows developer workstations to be hardened through minimal/read-only root host OSes by removing risk and complexity into containers - the same benefits in robustness and management as containerizing cloud workloads.

The convergence of projects like Fedora/CentOS Atomic Host has taken this one step further, by sharing the same base OS and infrastructure between cloud and workstation use cases, you can bring the developer even closer to the production environment and easily test/run/develop infrastructure like Kubernetes alongside your desktop and development containers.

Core infrastructure such as the bubblewrap sandboxing tool and OStree system image management library are used by Flatpak and shared with Project Atomic.

Any other relevant details we should know about while preparing the infrastructure?

Flatpak uses bubblewrap to set up the containers for the builds, which (on most kernels) requires a privileged helper to set up a new user namespace. Typical container infrastructure doesn't allow priveleged operations like this, meaning Flatpak builds would fail at present. We'd be interested in learning if we can use cloud tech such as Kubernetes to run the Flatpak builds containerized and integrate it with our infrastructure that way - it should make the management/provision of instances more standardised and portable - if we can understand/solve the privilege issue. It would be for "Version 2" of our setup.

[idvoretskyi]

@dankohn please, take a look.

[dankohn]

+1. @taylorwaggoner will get you set up.

[ramcq]

Great news, thanks @dankohn @idvoretskyi. Do you have any pointers about the nested containers thing? Anyone who's used bwrap inside a kubernetes container?

[dankohn]

@idvoretskyi can point you to the Slack channel where you have the best shot of getting an answer.

[taylorwaggoner]

@ramcq - I've set this project up in Packet and have sent you an invitation to join. Let me know if you have any questions.

[idvoretskyi]

@ramcq no problem, feel free to DM me at https://cloud-native.slack.com.

[jacobsmith928]

@ramcq I wanted to check in on this request and ongoing usage. Looks like you have a decent amount of infra (3x large x86 and 3x large Arm) and I wonder if anything can be consolidated.

@taylorwaggoner can you and @vielmetti please review as well?

[ramcq]

@jacobsmith928 One of the large x86 is a test system we can decommission for the time being, the others are all in a pool for builds for https://flathub.org/builds/#/workers. I think we could take one of the ARM builders off and see whether we can keep up with 2x2 - would that be OK?

\cc @barthalion @alexlarsson

[barthalion]

I have deleted 1x t1.small.x86, 1x c1.large.arm.xda and 1x m1.xlarge.x86.

[jacobsmith928]

Great tahnks @barthalion - appreciate it!

[barthalion]

Hey @jacobsmith928, would it be a problem if we add one c1.large.arm in AMS? I suspect our main load balancers hosted elsewhere are overloaded which causes increased 503 error rate, I'd like to check if it gets any better if we add new server to the pool.

[jacobsmith928]

@vielmetti can you please coordinate as you see fit?

[vielmetti]

Hi @barthalion - have added a c1.large.arm reserved server in our AMS1 facility. You'll want to provision it per our reserved hardware setup info:

https://www.packet.com/developers/docs/getting-started/deployment-options/reserved-hardware/

[barthalion]

Thanks a lot!

[barthalion]

Hi @vielmetti, our load balancers have been upgraded to use proper SSDs and the performance issues are gone now, so the c1.large.arm machine can be reclaimed. Thanks again!

[vielmetti]

Thanks @barthalion - that system has now been reclaimed.

[vielmetti]

@barthalion and all -

The current Flathub infrastructure uses two of a server type (c1.large.arm.xda) which we are removing from service. We should have a similarly equipped alternative arm64 system (c2.large.arm) to put in its place. I've flagged this project for review and will work with you on a swap.

[barthalion]

Hey Ed, any idea if Ubuntu or more generally non-CentOS distributions boot fine on c2.large.arm at this point?

[vielmetti]

@barthalion - we did a round of firmware updates on the c2.large.arm and as best I know we're back to 100% on OS booting. Of course we wouldn't take away the old servers until the new ones were up and stable.

[ramcq]

Are the new boxes capable of running 32 bit ARM binaries?

25 Oct 2021 14:54:01 Edward Vielmetti @.***>:

@barthalion[https://github.com/barthalion] - we did a round of firmware updates on the c2.large.arm and as best I know we're back to 100% on OS booting. Of course we wouldn't take away the old servers until the new ones were up and stable.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub[https://github.com/cncf/cluster/issues/56#issuecomment-950952938], or unsubscribe[https://github.com/notifications/unsubscribe-auth/AAB6EBB4IQ7Z5AKHK45UPOLUIVOPPANCNFSM4EQP7O5Q]. Triage notifications on the go with GitHub Mobile for iOS[https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675] or Android[https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub]. [data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAEIAAABCCAYAAADjVADoAAAAAXNSR0IArs4c6QAAAARzQklUCAgICHwIZIgAAAAnSURBVHic7cEBDQAAAMKg909tDwcUAAAAAAAAAAAAAAAAAAAAAPBjRFIAASHKmKkAAAAASUVORK5CYII=###24x24:true###][Tracking image][https://github.com/notifications/beacon/AAB6EBAA733E6XI7HVYQS63UIVOPPA5CNFSM4EQP7O52YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOHCXGH2Q.gif]

[vielmetti]

@ramcq the c2.large.arm will run 32-bit Arm binaries, yes. The usual conditions apply - the systems won't boot a 32-bit kernel, but they will run 32-bit apps e.g. in containers.

[barthalion]

Hey @vielmetti, any idea what has happened to sources.flathub.org? It hasn't been actively used lately, but it was definitely up, but I can't find it in Equinix Metal dashboard anymore.

[github-actions[bot]]

Checking if there are any updates on this issue