Merge with Debian OBS?

[ghost]

I just noticed this was a thing now. Should we merge the OBS repository with the one I setup for Debian?

[ghost]

I'm also thinking of seeing if regular user accounts can help manage the repository since the automated processes are mainly needed for uploading new builds.

[wchen342]

I am good with moving the repo under the one you created, but I don't know how it affect the workflow. Currently I just test everything out in a branched -dev repo (I do not build locally) then submit the package to the actual public repo. Will the branched repo be kept after the movement?

I am not really familiar with how _service works and the OBS documentations are kind of messy, so I currently upload everything manually. If you can help with that it will be great.

[ghost]

I'll have to look into how it works for RPM systems specifically. But currently I have it setup based on the structure of the Debian repos. For Fedora or CentOS, each distribution variant would need its own branch in this repository which would then upload to a specific package in either the production or development project. I am currently working on seeing if I can finish the Arch support first since it should be the simplest.

[wchen342]

For Fedora or CentOS, each distribution variant would need its own branch

Yeah that will be better. For upstream (the fedora chromium package) usually they will merge modifications on each distribution into master so eventually all branches will be level with the main branch (because spec files have conditions that can change the building process based on the target distribution being run on), but separate branches/packages can deal with some specific situations better I guess.

[ghost]

Yes, we frequently have to use patches specific to distribution releases. You can try to come up with another option but you still need something to signal the differences between builds. If you can create a single set of files that will build correctly for all Fedora / CentOS environments then we might be able to make that work. The hack I came up with was to compensate for the differences between the branches being generally incompatible.

[ghost]

In this regard it's possible RPM is more sensible than Debian's whole system as Debian often ends up generating source packages specific to a release.

[ghost]

Huh. It seems that's what you already do? That should make implementation much easier.

@Eloston Can I get commit access to this repository so I can implement OBS support?

[wchen342]

It seems that's what you already do?

You mean the spec file or the OBS repo? For the spec file you can have a look at lines like this where conditions are used for applying different things on different distributions.

For the OBS repo I just activate 4 distributions on the Repositories tab in the project (and add some sources because some non-free yum (=apt on Debian) sources are not activated by default).

[ghost]

I meant you're able to just use a single package for the whole Fedora distribution set. Here I have to use a separate package for each.

https://build.opensuse.org/project/show/home:ungoogled_chromium

[wchen342]

Ok yes that is benifited from the spec file's ability of doing conditioning branches, so everything can be put in a single repo. Guess Debian doesn't allow that?

[ghost]

Not really. It seems to expect them to be generated from a source repository or so. We can't selectively enable portions, etc.

[ghost]

I have created a PR for some stuff I found while researching this problem. See #3.

I noticed from the sources that you are currently uploading a custom modified tarball. We had to do something similar for Debian before and it was a hassle to get it working well for OBS. We ended up having to extract the regular tarball and apply the modification scripts prior to the build.

Is this an option we might be able to use for Fedora? In any case it would mean changes to the existing build scripts to facilitate it. Otherwise our script for OBS would probably have to repackage it prior to upload. If that is the case, how is it done?

[wchen342]

I have created a PR

LGTM, merged.

We ended up having to extract the regular tarball and apply the modification scripts prior to the build.

Is that allowed by OBS? Fedora build servers and COPR doesn't allow upload of source codes that contains ffmpeg with proprietary codecs, I thought OBS has the same policy? If the upload is allowed then sure we can do that in the building script.

If that is the case, how is it done?

It is done by chromium-latest.py --version {version_of_chromium} --ffmpegclean (--ffmpegarm). The script will download original chromium source and clean ffmpeg, then repack it automatically.

[ghost]

Well, we upload the original chromium tarballs for both the Debian and Arch repositories already. We don't repack them prior to upload. And I looked at the official OpenSUSE packaging for chromium. I don't see any attempt made to do this in their scripts. I suspect we'd be OK if we just used the Fedora system ffmpeg and let it be at that. Our final binaries would probably not include the codecs they have an issue with as we would be using the distribution ffmpeg and not the bundled one.

https://build.opensuse.org/package/view_file/network:chromium/chromium/chromium.spec

Your thoughts, @Eloston?

[wchen342]

What exactly was the problem with repacking before upload?

I would prefer either repackaging before uploading for two reasons:

• It is a one liner with chromium-latest.py
• I want to avoid unnecessary modifications on the spec file so it is easier to follow up with upstream updates

Or the other option is do not upload anything at all but run chromium-latest.py directly in spec file, which shall also be one or two lines change.

we upload the original chromium tarballs for both the Debian and Arch repositories already

That means probably OBS is not as strict about this as Fedora build servers.

The upstream packaging uses bundled ffmpeg for a reason. ffmpeg doesn't come with default repositories on Fedora and CentOS. On Fedora it is in RPMFusion-Free, and on CentOS it is in EPEL. I checked and all other dependencies are in default repos, which means adding ffmpeg as a dependency will require users to enable extra repositories.

[ghost]

The main problem is it slows down the github action a fair bit. I didn't know there was more involved than just Fedora's policies. I will see what I can do to just automate the current method then.

[ghost]

I've been leveraging _service for the other setups to make OBS download the source files instead of us needing to do so as a performance enhancement. It's expensive to upload large packages like this. But we can't really do that if we need to customize the source files prior to upload.

[wchen342]

In that case I think it is better to run chromium-latest.py in either _service if possilble, or run in spec file directly.

[ghost]

I'll see if it's possible to prepare the sources immediately before our build like we do with Debian.

[ghost]

If we do make these changes we can hide them behind an OBS variable so it won't be enabled for manual builds.

[ghost]

Looks like Chromium 90 is due to land soon. I'll wait on finishing up the OBS support here.

[ghost]

It's taking longer than expected so I have gone ahead and implemented it. I am still waiting for the builds resulting from my test upload to see if they build correctly or not. I have put up a proposal in the PR. It works via repository secrets so I will still need access to finish the last part. But this should work now. You can also use it to perform scripted manual uploads provided they conform to the same structure.

[ghost]

I ended up needing to use rpmspec to extract all the information I needed to generate the _service file. I can't use the download_files method because we rely on conditional source files so I am forced to generate all the download URLs by simulating each of the supported release targets in order to get the complete list.

[ghost]

After tinkering for awhile I have figured out what was keeping some of the repositories from building. It's ready to be merged whenever though I find I am going to have to use some project configuration for centos 8.

[ghost]

Currently seeing if I can get it to build for i586 and aarch64 since those architectures are mentioned in the packaging.

Edit: Seems i586 has issues so I am going to leave that disabled. aarch64 should be OK. 32 bit x86 isn't in as much demand these days so it's probably best left out.

[ghost]

So far it's looking promising. My workaround appears to have worked. But it's going to be a day or so before I know if this is going to work for aarch64 due to how slow it is. If it does I'll do a repeat test of it with my official changes to make it work for OBS.

[ghost]

After that the only thing left before I can make it automated is regular access to this repository.

[Eloston]

@braewoods Access should have been granted several days ago, let me know if this is not the case.

[ghost]

@Eloston It hasn't been. I'm a member of the team but for some reason I have no special privileges to this repository. Or at least I don't have access to the repository secrets.

[Eloston]

@braewoods Right, I think I gave the fedora team Maintain access only. I've granted you and @wchen342 Admin access.

[ghost]

Ok, seems everything is in order to deploy OBS. I'll test it shortly.