MesserLab / SLiM

SLiM is a genetically explicit forward simulation software package for population genetics and evolutionary biology. It is highly flexible, with a built-in scripting language, and has a cross-platform graphical modeling environment called SLiMgui.
https://messerlab.org/slim/
GNU General Public License v3.0
161 stars 31 forks source link

Copr Integration for RPM Packaging CI/CD #174

Closed bryce-carson closed 2 years ago

bryce-carson commented 3 years ago

@bhaller, if we determine how to use the Commitish field to specify tags, we can use a Copr webhook. I don't know how Commitishes work, so how you would specify a tag like I'm not sure, is it a regex like v[[:digit:]](.[[:digit:]])*?

SCM

This method allows you to build RPM(s) from any Git, DistGit, or SVN repository containing a valid .spec file. The only required argument is Clone URL and if the target repository places the .spec file together with package sources in the root directory and you want to build from master HEAD, it will simply work. There are more things to configure for more complex cases. E.g. you might want to specify Subdirectory of the target repository if that is where the repository has the package sources placed. See the following list for the full option description:

  • Type: SCM type of the repository being pointed to by Clone URL (in other words, whether we should use plain git or git svn for subsequent cloning).
  • Clone URL: What repository we should clone to obtain the sources.
  • Committish: What tag, branch, or commit we should check out from the history of the cloned repository. By default HEAD of master branch.
  • Subdirectory: Where the subsequent SRPM build command (see below) should be executed. The path is relative to the repository root.
  • Spec File: Path to the spec file relative to the given Subdirectory. Note that you can optionally anchor the path with / (e.g. /rpm/example.spec). If not specified the .spec file will be auto-located.

Using this, we can automatically trigger builds for the HEAD of the master branch, by defining a new package SLiM-testing with the SCM package type on Copr. Then the status badge on the README will actually reflect the CD status of the repository.

We can also use this method for future tagged releases.

This requires a PR of the RPM .spec file to the repository, and for you to configure the web hook on the GitHub repository settings. You've got administrator access to the Copr repository, so if you want to look at this you should be able to go through the process yourself. If you need my help, however, I am available.

The following files would need to be in the main part of the repository for a simple configuration, but if you'd like you can use a Copr subdirectory and place some files there if you tell Copr where to find them and modify the RPM spec file a little.

filesForCoprIntegration.zip

bhaller commented 3 years ago

Hi Bryce. I'm afraid I don't really understand what you're talking about here, sorry. I've never heard of Committish, don't know what a Copr webhook is, etc. Can you clarify?

bryce-carson commented 3 years ago

This is a similar setup to Travis CI. Configuring this allows you, or anyone with commit permissions to the SLiM repository, to trigger automatic builds for the Fedora-family and SUSE-family of Linux distributions; similar to what happens when you make a commit and Travis tests the build on Mac OS and Ubuntu. I believe the SLiM repository has this setup through GitHub Actions for Travis CI.

For an explanation of what a Commitish is, this quotation from the Git glossary may or may not suffice:

commit-ish (also committish) A commit object or an object that can be recursively dereferenced to a commit object. The following are all commit-ishes: a commit object, a tag object that points to a commit object, a tag object that points to a tag object that points to a commit object, etc.

For an explanation of what a Webhook is, this quotation from the Copr Documentation on GitHub Webhooks is helpful:

GitHub Webhooks Webhooks allows you to automatically trigger build. First you need to go to your Copr project and tab “Packages” and define some package. The only source type which make sense together with webhooks is “SCM”. Check the “Webhook rebuild” option. You may hit “rebuild” and test the build actually works. Now you can navigate to “Setting” tab and then “Webhooks” There is your webhook url in the form of https://copr.fedorainfracloud.org/webhooks/github/<ID>/<UUID>/. Then in your GitHub project, go to Settings / Webhooks and services. Click on the Add webhook button. Fill in the Payload URL field with the url you noted previously. Set the other fields to the values: content: application/json; send just push event; no secret. Click the Add webhook button. And next time you push anything to your git, Copr will automatically rebuild your package.

This can be configured for a new package on my Copr repository for SLiM, with the suggested name of SLiM-testing. This will create a new package in the repository, which the end-user will not download or install (the SLiM package is not the SLiM-testing package).

I'll describe it as a process as well: 1) A new commit is made to the master branch 2) GitHub informs Copr that a commit was made 3) Copr downloads the HEAD of the repository's master branch 4) Copr builds the sources according to SLiM-3.6.spec 5) Copr notifies GitHub if a build was successful or unsuccessful

bryce-carson commented 3 years ago

This depends on my creating a new RPM spec file in my fork. I'll work on that as I am able, but once it is setup automotic builds for testing the current source HEAD should be breezy. :octocat:

bhaller commented 3 years ago

Hi Bryce. Hmm. OK, so if I'm understanding correctly, the purpose of this is to provide another platform for CI testing, based on Copr on Fedora. I appreciate your thinking on this, but I'm not convinced that the value-add here is worth the additional complexity for the project setup. So far it feels like the GitHub Actions CI, combined with my own testing on macOS, has sufficed. I'm just not sure that we need CI on another platform. IIRC we haven't seen any build issues that have been Fedora-specific, have we? Do you think we really need this? I think I'd prefer to keep things simple, at least until an actual need for this becomes clear. I think I'd propose that we keep this issue open, as a placeholder, but not act on it for the time being. What do you think?

bryce-carson commented 3 years ago
There were a couple Fedora-specific build issues, I believe: Issue Fedora Version Commit
Unicode Pragma Mark Fedora 32 MesserLab/SLiM@ce7cba200c8c7af53b679218f6f43d2d5c39150e
Additional #include needed Fedora 34 MesserLab/SLiM@7964f7911e1665a1ba0783d9136b3d3bff06d931

Those are the only two I can recall or am aware of, so I don't think it is needed, but it's an option that exists to ensure that nothing sneaks in that would prevent compilation in the future. If you'd like to leave it, we can keep the issue open and you may decide in the future if it's worth setting up.

bhaller commented 3 years ago

I guess those were so minor that I forgot about them. Hmm. Let's leave it for now and I'll think about it. Thanks for the proposal; if more issues arise in future perhaps we'll take it up, good to have the option.

bhaller commented 3 years ago

Hi Bryce. Reviewing old issues. If I understand this correctly, once set up it would operate automatically whenever a commit is pushed to the GitHub repo, and I'd just get an email saying that it succeeded or failed, and I could look at the logs for the build if it failed. And if this particular testing system breaks down, it wouldn't hurt anything else; the existing GitHub Actions tests would not be dependent upon this new stuff in any way. If those things are true, then I guess maybe this would be a good thing to have; another testing platform can't hurt! :-> Now that I've gotten finished with some other big things, I'm feeling more able to take little things like this on, so go ahead with this if you still think it's a good idea. Thanks!

bryce-carson commented 2 years ago

Hi Ben,

I'm closing this issue as I don't think we will need the extra build information. If something happens with the C++ or Qt5 sources that really bogs SLiM's build, I'm sure we'll find out soon enough, and it won't affect releasing.

It's an extra complication that I don't think we need at this time, or in the future.