hdl / containers

Building and deploying container images for open source electronic design automation (EDA)
https://hdl.github.io/containers/
Apache License 2.0
106 stars 24 forks source link

Coordination/reuse of copr (Fedora/CentOS) projects #41

Open umarcor opened 2 years ago

umarcor commented 2 years ago

https://twitter.com/mithro/status/1426666143914217474

Check this out @mithro / @unaimarcor … we could base the HDL images on CentOS and use the already existing pre-built packages. Cristian is also adding ppc64le so we got the three major archs going on (x64, arm64, ppc64le). When riscv gets a stable image, it could be added too.

https://twitter.com/cbalint13/status/1426624389840257030

It can be, will do it right now for all future builds on weekly basis. Also will add ghdl & ghdl-yosys-plugin. You can also check VLSI packages (OpenLANE is WiP): https://t.co/RPJCii69Kk?amp=1

https://twitter.com/mithro/status/1426666143914217474

I believe @unaimarcor has CentOS based images for the HDL/containers too?

Since some communities use (need) CentOS based images, some months ago I added support for multiple "collections" in this repo; those being "debian/buster", "centos/7" and now "debian/bullseye". However, taking care of the Debian collections is already enough effort for myself, so I did not port all the dockerfiles to centos/7 (just base.dockerfile).

Actually, I am familiar with Fedora and dnf (that's the Linux desktop on my laptop and workstation), so I could help with that if some contributor wants to step in. But I don't have any motivation for dealing with Centos on my own and I found Fedora containers to be consistently larger (and slower) than Debian for the same purpose. I do use CentOS and Fedora containers for testing (see e.g. https://github.com/ghdl/docker/actions/runs/1134401041), but not for using the tools in CI.

Note that a different use case would be providing images here with build dependencies, which developers of OpenROAD, OpenLANE, VTR can use in their internal CI. That's not in the scope at the moment, but we might just publish those images (because we are using them internally).

With regard to https://copr.fedorainfracloud.org/coprs/rezso/VLSI/, I'm happy to see effort for having EDA tools packaged in rpm. There was an "spin" named Fedora Electronic Lab some years ago. That was very interesting because several EDA vendors did only support Red Hat officially, so it provided a nice alternative that was almost guaranteed to work and it provided lots of interesting open source EDA. I believe that "spins" were renamed to "labs" (https://labs.fedoraproject.org/es/), but the Electronic one did not survive.

The unfortunate part is that creating a project in copr is not different from creating a repo in GitHub. I have a profile there (https://copr.fedorainfracloud.org/coprs/umarcor) and I evaluated it for GHDL. Both copr and Actions provide CI based on "containers", which can be used for building rpm packages for multiple distributions. Therefore, the complexity is similar: writing and maintaining the (rpm) recipes. Hence, the challenge is fragmentation. A quick search for "yosys" reveals that:

Conversely, in this repo, we provide more than 20 tools (and the list is increasing). We want to provide a consistent environment for using all or any of them, so that users can have the benefits of lightweight granular images but the sense of being on the same context. Similarly, conda-* and MINGW-packages support "a lot" of tools.

Therefore, rather than picking tools from copr repos "arbitrarily", it would be desirable if one copr project decided to cover all EDA tools; in coherence with this repo, MINGW-packages, conda-eda, etc. In the mid term, the purpose of that copr repo should be to coordinate/communicate with Fedora maintainers. Precisely because the Electronics Labs spin existed, there is a background of maintainers which keep some EDA tools up to date. There are issues that might prevent some tools from being upstreamed, thus the convenience of having one copr repo as an entrypoint. Should no one be willing to coodinate a single copr project, we might create hdl/rpm for coordination purposes.

With regard to reusing the content in this repo, coprs/rezso/VLSI uses pinned versions of the tools:

That is inconsistent with other tools in this repo. Hence, we might create collections "centos/8" and "fedora/35", copy the recipes and modify them so they use the master/main branches. Alternatively, the recipes in rezso repo might be modified (or duplicated). In ArchLinux, it is common to provide *-git variants of the packages. If they built bleeding-edge versions, we might consume their packages directly.

However, I am hesitant to create two new collections for a handful of new tools. There needs to be some willingness to port other tools, or otherwise contribute those to (one of) the Debian collections. Moreover, testing CentOS/7, CentOS/8, Fedora/35, etc. might make sense in copr; but here, we should stick to one collection for each "family of distributions", unless we get a bunch of active contributors. Currently we have debian-buster and debian-bullseye because we are in a transition/testing period (Bullseye was released last weekend). I expect to deprecate and stop building Buster images soon.

I believe the most desirable approach would be to have a copr repo and/or hdl/rpm, and enhance the documentation here to explain how users can create a custom container with packages from there.

umarcor commented 2 years ago

FTR, a lot of new packages were added to https://copr.fedorainfracloud.org/coprs/rezso/HDL/packages/ in the last days. There are over a dozen tools now.

cbalint13 commented 2 years ago

Hi @umarcor , @mithro

First thank you much for your time and effort picking the subject up !

Wasn't aware of this thread, until now, I also opened a small enlisting request here #3.

Allow me few more details on why this COPR repo exist:

My sole purpose was to automate a bit and publish things I gathered over time, copr is free and ideal for doing this, and other people may also benefit.

About quality: WorksForMe®, the effort is to make "latest" but also "complete" tools (see e.g. nextpnr having all families of FPGAs even the upcoming fpga-interchange)

I agree with @umarcor, COPR is no different to GITHUB, fragmentation & forking can occur.

As observation, at this moment, all other COPR repos holding EDA (i.e. take yosys) looks unmaintained.

If anyone wants to co-maintain or find a better home (perhaps more official) for these RH/Fedora related things I am all-in and very happy to help with the process as my time permits.

~Cristian.

cbalint13 commented 2 years ago

With regard to reusing the content in this repo, coprs/rezso/VLSI uses pinned versions of the tools:

Hi @umarcor, @mithro,

Of course releng world is never perfect, will try keep improve there things or issues as they arise.

Thank you much ! ~cristian.