Enable copr builds and add packit config

[dhodovsk]

Let us introduce packit service to you - the automation to integrate upstream open source projects into Fedora operating system.

After merging this PR, you are just a few steps away from RPM builds being automatically triggered on your PR's. It means, that you'll be able to try and play with your change, packaged as an RPM.

But there is more. By using packit, you can for example enable adding new releases into Fedora Rawhide.

What are the next steps?

• Install Packit-as-a-Service github application in your repo
• In case you are first user, wait for account approval
• Enjoy the built RPMs!

For more info, please:

• check out the documentation: https://packit.dev/docs/
• contact @packit-service/the-packit-team

The spec you see in this PR was fetched from your package's Fedora dist-git.

[dcantrell]

How does this work with dist-git for Fedora? Are builds still required there? This doesn't appear any different than setting up Copr to build on a PR webhook in github. If it will do builds in dist-git for me, that is an advantage.

Some other problems I see or would like clarified:

1) The .packit.yaml file explicitly lists Fedora releases and architectures to build for. This looks like something that requires maintenance. It would be preferable to just build for everything available for Fedora in Copr. All currently available releases and architectures. I don't want to have to maintain this list in .packit.yaml.

2) This PR also adds in pyparted.spec from dist-git in Fedora, which is something I explicitly removed from the upstream project because the packaging spec file is a downstream integration problem. I am not opposed to having a spec file template generated as part of making a new release which will update the relevant parts of the spec file. This needs to happen for packit too because I'm not going to maintain the spec file in two places. The most important aspects are changing the Version tag to match the release and generating the %changelog content automatically.

[dhodovsk]

How does this work with dist-git for Fedora? Are builds still required there? This doesn't appear any different than setting up Copr to build on a PR webhook in github. If it will do builds in dist-git for me, that is an advantage.

Building in copr is one of the multiple current and planned features of packit. Right now we support copr_build and propose_update. propose_update for every upstream release creates PR in dist-git.

packit can trigger tests of the RPM's after the copr build is finished using testing farm

In the future we plan to improve package integration into fedora and RHEL by supporting other workflows/jobs - build on every closed PR in dist-git, syncing downstream changes to upstream, etc.

Some other problems I see or would like clarified:

1. The .packit.yaml file explicitly lists Fedora releases and architectures to build for. This looks like something that requires maintenance. It would be preferable to just build for everything available for Fedora in Copr. All currently available releases and architectures. I don't want to have to maintain this list in .packit.yaml.

We now consider two ways of solving this problem. The first one is using aliases: https://github.com/packit-service/packit/issues/540, the other is to use all supported fedora targets by default: https://github.com/packit-service/packit/issues/574

2. This PR also adds in pyparted.spec from dist-git in Fedora, which is something I explicitly removed from the upstream project because the packaging spec file is a downstream integration problem. I am not opposed to having a spec file template generated as part of making a new release which will update the relevant parts of the spec file. This needs to happen for packit too because I'm not going to maintain the spec file in two places. The most important aspects are changing the Version tag to match the release and generating the %changelog content automatically.

Spec file presence in upstream is an issue that gets mentioned often. Please see if this can help: FAQ.

[dcantrell]

How does this work with dist-git for Fedora? Are builds still required there? This doesn't appear any different than setting up Copr to build on a PR webhook in github. If it will do builds in dist-git for me, that is an advantage.

Building in copr is one of the multiple current and planned features of packit. Right now we support copr_build and propose_update. propose_update for every upstream release creates PR in dist-git.

packit can trigger tests of the RPM's after the copr build is finished using testing farm

In the future we plan to improve package integration into fedora and RHEL by supporting other workflows/jobs - build on every closed PR in dist-git, syncing downstream changes to upstream, etc.

Those are nice, but it still didn't answer my question. Do I still need to build releases of pyparted in dist-git manually or does packit take care of that?

Some other problems I see or would like clarified:

1. The .packit.yaml file explicitly lists Fedora releases and architectures to build for. This looks like something that requires maintenance. It would be preferable to just build for everything available for Fedora in Copr. All currently available releases and architectures. I don't want to have to maintain this list in .packit.yaml.

We now consider two ways of solving this problem. The first one is using aliases: packit-service/packit#540, the other is to use all supported fedora targets by default: packit-service/packit#574

Both of these options are fine with me. I would like to see the PR modified to do this.

1. This PR also adds in pyparted.spec from dist-git in Fedora, which is something I explicitly removed from the upstream project because the packaging spec file is a downstream integration problem. I am not opposed to having a spec file template generated as part of making a new release which will update the relevant parts of the spec file. This needs to happen for packit too because I'm not going to maintain the spec file in two places. The most important aspects are changing the Version tag to match the release and generating the %changelog content automatically.

Spec file presence in upstream is an issue that gets mentioned often. Please see if this can help: FAQ.

For that to work, pyparted needs to gain a template spec file. I don't do that right now because pyparted is just me and I keep the spec file in dist-git. If you want to do this, I would like to see a separate PR that first adds a template spec file. This spec file should:

• Carry a replaceable variable for the version
• Carry a replaceable variable for the source archive name
• Generate a block for the %changelog and add that

You should be able to generate a spec file as part of the release process when a tarball is generated. Or separately.

After that PR, modify this PR to use the spec file template per packit's instructions. For an example of what I have done for a spec file template, see: https://github.com/rpminspect/rpminspect

[dhodovsk]

How does this work with dist-git for Fedora? Are builds still required there? This doesn't appear any different than setting up Copr to build on a PR webhook in github. If it will do builds in dist-git for me, that is an advantage.

Building in copr is one of the multiple current and planned features of packit. Right now we support copr_build and propose_update. propose_update for every upstream release creates PR in dist-git. packit can trigger tests of the RPM's after the copr build is finished using testing farm In the future we plan to improve package integration into fedora and RHEL by supporting other workflows/jobs - build on every closed PR in dist-git, syncing downstream changes to upstream, etc.

Those are nice, but it still didn't answer my question. Do I still need to build releases of pyparted in dist-git manually or does packit take care of that?

Right now only by using packit CLI: https://packit.dev/docs/cli/build/

Some other problems I see or would like clarified:

1. The .packit.yaml file explicitly lists Fedora releases and architectures to build for. This looks like something that requires maintenance. It would be preferable to just build for everything available for Fedora in Copr. All currently available releases and architectures. I don't want to have to maintain this list in .packit.yaml.

We now consider two ways of solving this problem. The first one is using aliases: packit-service/packit#540, the other is to use all supported fedora targets by default: packit-service/packit#574

Both of these options are fine with me. I would like to see the PR modified to do this.

Of course, we provided the links to issues so feel free to subscribe.

1. This PR also adds in pyparted.spec from dist-git in Fedora, which is something I explicitly removed from the upstream project because the packaging spec file is a downstream integration problem. I am not opposed to having a spec file template generated as part of making a new release which will update the relevant parts of the spec file. This needs to happen for packit too because I'm not going to maintain the spec file in two places. The most important aspects are changing the Version tag to match the release and generating the %changelog content automatically.

Spec file presence in upstream is an issue that gets mentioned often. Please see if this can help: FAQ.

For that to work, pyparted needs to gain a template spec file. I don't do that right now because pyparted is just me and I keep the spec file in dist-git. If you want to do this, I would like to see a separate PR that first adds a template spec file. This spec file should:

• Carry a replaceable variable for the version
• Carry a replaceable variable for the source archive name
• Generate a block for the %changelog and add that

You should be able to generate a spec file as part of the release process when a tarball is generated. Or separately.

For spec file generation checkout the Universal packaging Tool that specializes in that.

After that PR, modify this PR to use the spec file template per packit's instructions. For an example of what I have done for a spec file template, see: https://github.com/rpminspect/rpminspect

For your case, I'd suggest downloading the specfile template and then process it in a make action or another command such as for example in https://github.com/cockpit-project/cockpit-podman/blob/master/packit.yaml

More info about actions can be seen here: https://packit.dev/docs/actions/

[dhodovsk]

Of course, sorry about that. I can make the changes, set the actions for spec file generation in packit and make the changes in .packit.yaml when issues below are fixed. However, I will not be the right person for creating/managing the spec file template. Would you be willing to help with that file or point me to a person that could when all other blockers are resolved?

For summarisation blocking issues on our side are 1, missing support for building in fedora - https://github.com/packit-service/packit-service/issues/46(packit-service to react to new commits in a dist-git branch and be able to do builds) when combined with already supported propose_downstream, it will build in fedora on release tag in upstream 2, specific fedora targets not required - use aliases (current/last/supported) - https://github.com/packit-service/packit/issues/540

Is there anything else that I forgot or anything that you'd consider a blocker?

[dcantrell]

Of course, sorry about that. I can make the changes, set the actions for spec file generation in packit and make the changes in .packit.yaml when issues below are supported. However, I will not be the right person for creating/managing the spec file template. Would you be willing to help with that file or point me to a person that could when all other blockers are resolved?

I do not have time to work on this. One of my other projects already contains a spec file template as I have described and it's generated for Copr builds and releases. See https://github.com/rpminspect/rpminspect

This is similar to what I was describing. The missing piece is automatic generation of %changelog entries. I have begun work on that for rpminspect, but it's not complete. There's also the aspect of pyparted already having a spec file %changelog and I would prefer to preserve that going forward with new releases automatically generating it from git. This is not difficult, I just don't have time to work on it for pyparted at the moment.

For summarisation blocking issues on our side are 1, missing support for building in fedora - packit-service/packit-service#46 to react to new commits in a dist-git branch and be able to do builds) when combined with already supported propose_downstream, it will build in fedora on release tag in upstream 2, specific fedora targets not required - use aliases (current/last/supported) - packit-service/packit#540

Is there anything else that I forgot or anything that you'd consider a blocker?

Not that I can think of. These are the critical aspects to make packit useful to me. The main points are:

• I never want to modify .packit.yaml once it's committed.
• I want new builds to happen in Fedora when a release tag is made. I do not care about packit builds happening on PRs. I already take care of that through services like Travis-CI and Copr.
• I never want to maintain a spec file for each release. Whatever is required in a spec file for a release should be automatically generated.

Oh, I can think of one thing. Packit should do builds in Fedora from a new tag if associated CI is passing. In my case, I want it to build only if Travis-CI passes on the release tag.

[TomasTomecek]

Dominika asked to me to reply here since I'm back from my vacation now.

This is similar to what I was describing. The missing piece is automatic generation of %changelog entries. I have begun work on that for rpminspect, but it's not complete. There's also the aspect of pyparted already having a spec file %changelog and I would prefer to preserve that going forward with new releases automatically generating it from git. This is not difficult, I just don't have time to work on it for pyparted at the moment.

So the expectation is that when a new release happens (= new git tag) some automation would create a new changelog entry. What should be the content? Just new upstream release: 1.2.3 or an actual list of features? Should the %changelog be stored in the upstream git or generated on the fly and be only applied in Fedora?

Oh, I can think of one thing. Packit should do builds in Fedora from a new tag if associated CI is passing. In my case, I want it to build only if Travis-CI passes on the release tag.

This is really interesting suggestion. I wonder if GitHub has an API for this.

Thanks David, I can see a bunch of interesting RFEs here: we'll make sure to discuss them within the team.

[dcantrell]

Dominika asked to me to reply here since I'm back from my vacation now.

This is similar to what I was describing. The missing piece is automatic generation of %changelog entries. I have begun work on that for rpminspect, but it's not complete. There's also the aspect of pyparted already having a spec file %changelog and I would prefer to preserve that going forward with new releases automatically generating it from git. This is not difficult, I just don't have time to work on it for pyparted at the moment.

So the expectation is that when a new release happens (= new git tag) some automation would create a new changelog entry. What should be the content? Just new upstream release: 1.2.3 or an actual list of features? Should the %changelog be stored in the upstream git or generated on the fly and be only applied in Fedora?

I want to be able to trigger a package build when I make a new release as a tag. What that should do is:

• Generate a spec file from the spec file template in the source repo
• Generate a tarball of the release using git archive
• Do the work necessary to build the release in dist-git for Fedora

For me a spec file template would have three things changed:

• The Version: tag
• The Source0: archive
• The %changelog block

The changelog gets tricky because it needs to live somewhere. I don't want to store it in the upstream repo because that information is really only relevant for the downstream packaging effort. Here's what I would suggest:

• Generate a new %changelog block of changes since the last release. I expect this to be the git log output like this: git log --format=%s PREVIOUS_TAG..
• When generating the new spec file, it should combine the already existing spec file changelog data from the dist-git repo with the new data generated

The git log command should be wrapped so entries are prefixed and lines wrapped. This script gets you part of the way there: https://github.com/rpminspect/rpminspect/blob/master/utils/mkrpmchangelog.sh

The above information is what I want in an ideal world. The changelog data gets complicated and this may be too complicated for packit to support in a generic sense. At the very least, I want packit to be able to take a template spec file and update the Version and Source0 lines. In that case, then during my release process, I can generate the changelog data and add it to the spec file template. I don't actually like doing this because I only want the changelog data in the downstream repo. The main reason for that is that other people in Fedora may fix something in a package outside of the normal release process. That change will have entries in the changelog and I would expect those to be preserved. That's why I suggested the workflow above. Packit should generate the new changelog block and combine it with what exists in the downstream repo when generating a new spec file from the template.

Oh, I can think of one thing. Packit should do builds in Fedora from a new tag if associated CI is passing. In my case, I want it to build only if Travis-CI passes on the release tag.

This is really interesting suggestion. I wonder if GitHub has an API for this.

The Webhooks API can support this: https://developer.github.com/webhooks/

For rpminspect and pyparted, I use webhooks to trigger Travis-CI runs. For rpminspect, I trigger Copr builds on pushes so that new builds are continually available. You can select different events for the webhook, such as a release being made.

I have not looked, but I hope there is something in the webhook API where you can query if Travis-CI is passing for that release. Alternatively, the webhook could just run a new Travis-CI job and if that passes, do the packit run.

Thanks David, I can see a bunch of interesting RFEs here: we'll make sure to discuss them within the team.

Sure thing. If you want to discuss anything more, let me know.

[TomasTomecek]

Another super-late response from me since I was sick and now I'm finally replying to all the notifications.

I want to be able to trigger a package build when I make a new release as a tag. What that should do is:

* Generate a spec file from the spec file template in the source repo

* Generate a tarball of the release using git archive

* Do the work necessary to build the release in dist-git for Fedora

I feel like this is where we are heading with the packit project. Some parts can already be done now (1st item) while we are working on the third point, koji builds, in the coming sprint.

As for the second: creating a tarball is not a problem, but should the tarball be uploaded to some place? Also bear in mind that GitHub generates a tarball for every git tag.

For me a spec file template would have three things changed:

* The Version: tag

* The Source0: archive

* The %changelog block

The changelog gets tricky because it needs to live somewhere. I don't want to store it in the upstream repo because that information is really only relevant for the downstream packaging effort. Here's what I would suggest:

I'd say that changelogs are very relevant to upstream projects since it's a common place for an user or contributor to check for changes which happened in the last upstream releases.

* Generate a new %changelog block of changes since the last release.  I expect this to be the git log output like this:  git log --format=%s PREVIOUS_TAG..

* When generating the new spec file, it should combine the already existing spec file changelog data from the dist-git repo with the new data generated

Combining two changelogs sounds like a tough problem to solve. Wouldn't it be better to receive upstream pull requests whenever the spec file changes downstream?

The git log command should be wrapped so entries are prefixed and lines wrapped. This script gets you part of the way there: https://github.com/rpminspect/rpminspect/blob/master/utils/mkrpmchangelog.sh

The above information is what I want in an ideal world. The changelog data gets complicated and this may be too complicated for packit to support in a generic sense. At the very least, I want packit to be able to take a template spec file and update the Version and Source0 lines. In that case, then during my release process, I can generate the changelog data and add it to the spec file template. I don't actually like doing this because I only want the changelog data in the downstream repo. The main reason for that is that other people in Fedora may fix something in a package outside of the normal release process. That change will have entries in the changelog and I would expect those to be preserved. That's why I suggested the workflow above. Packit should generate the new changelog block and combine it with what exists in the downstream repo when generating a new spec file from the template.

I really like the changelog script you provided: I feel like we should give the power to all the upstream projects to create such script on their own and plug it into the packit workflow.

Oh, I can think of one thing. Packit should do builds in Fedora from a new tag if associated CI is passing. In my case, I want it to build only if Travis-CI passes on the release tag.

This is really interesting suggestion. I wonder if GitHub has an API for this.

The Webhooks API can support this: https://developer.github.com/webhooks/

For rpminspect and pyparted, I use webhooks to trigger Travis-CI runs. For rpminspect, I trigger Copr builds on pushes so that new builds are continually available. You can select different events for the webhook, such as a release being made.

I have not looked, but I hope there is something in the webhook API where you can query if Travis-CI is passing for that release. Alternatively, the webhook could just run a new Travis-CI job and if that passes, do the packit run.

There are webhooks for the check API, but afaik no one is using that interface - every CI system is setting commit statuses.

Not sure if commit comments webhook receives also the status updates.

[dcantrell]

Another super-late response from me since I was sick and now I'm finally replying to all the notifications.

Sorry to hear. I hope you are feeling better now.

I want to be able to trigger a package build when I make a new release as a tag. What that should do is:

* Generate a spec file from the spec file template in the source repo

* Generate a tarball of the release using git archive

* Do the work necessary to build the release in dist-git for Fedora

I feel like this is where we are heading with the packit project. Some parts can already be done now (1st item) while we are working on the third point, koji builds, in the coming sprint.

That is good to hear.

As for the second: creating a tarball is not a problem, but should the tarball be uploaded to some place? Also bear in mind that GitHub generates a tarball for every git tag.

I know that GitHub does this for every tag. I prefer generating an archive using 'git archive' locally. I also sign this file with gpg and upload both to a release announcement page. I try to do this consistently for my projects. I want the generated tarball and detached signature to actually be in the SRPM. I like having the ability to verify source archives and I know others do too.

For me a spec file template would have three things changed:

* The Version: tag

* The Source0: archive

* The %changelog block

The changelog gets tricky because it needs to live somewhere. I don't want to store it in the upstream repo because that information is really only relevant for the downstream packaging effort. Here's what I would suggest:

I'd say that changelogs are very relevant to upstream projects since it's a common place for an user or contributor to check for changes which happened in the last upstream releases.

Agreed. I am now automatically generating RPM changelog blocks using git log.

* Generate a new %changelog block of changes since the last release.  I expect this to be the git log output like this:  git log --format=%s PREVIOUS_TAG..

* When generating the new spec file, it should combine the already existing spec file changelog data from the dist-git repo with the new data generated

Combining two changelogs sounds like a tough problem to solve. Wouldn't it be better to receive upstream pull requests whenever the spec file changes downstream?

I would not want a pull request for a changelog update. Changelog changes are the #1 thing that cause merge conflicts. So I want to keep changelog updates in a separate commit from the functional changes. That leaves me the ability to do a 'git cherry-pick' to other branches and avoid merge conflicts on changelog changes.

I am doing this now for rpminspect using tools from my continuous-goodies project: https://github.com/dcantrell/continuous-goodies

The utils/mkrpmchangelog.sh script does the changelog part. If you look at the spec file and then in dist-git for rpminspect, you will see that I store the changelog in a separate file and the spec file does a %include on it. The changelog is listed as a Source1 file so the %include macro works easily.

This has it to where I can do "make release" in rpminspect to generate a new release on Github and "make koji" to generate new builds in dist-git.

The git log command should be wrapped so entries are prefixed and lines wrapped. This script gets you part of the way there: https://github.com/rpminspect/rpminspect/blob/master/utils/mkrpmchangelog.sh The above information is what I want in an ideal world. The changelog data gets complicated and this may be too complicated for packit to support in a generic sense. At the very least, I want packit to be able to take a template spec file and update the Version and Source0 lines. In that case, then during my release process, I can generate the changelog data and add it to the spec file template. I don't actually like doing this because I only want the changelog data in the downstream repo. The main reason for that is that other people in Fedora may fix something in a package outside of the normal release process. That change will have entries in the changelog and I would expect those to be preserved. That's why I suggested the workflow above. Packit should generate the new changelog block and combine it with what exists in the downstream repo when generating a new spec file from the template.

I really like the changelog script you provided: I feel like we should give the power to all the upstream projects to create such script on their own and plug it into the packit workflow.

That would work really well. Since I posted, I have updated it. See above. Everything is in my continuous-goodies project. Feel free to take it as an example, use it, or reference it.

Oh, I can think of one thing. Packit should do builds in Fedora from a new tag if associated CI is passing. In my case, I want it to build only if Travis-CI passes on the release tag.

This is really interesting suggestion. I wonder if GitHub has an API for this.

The Webhooks API can support this: https://developer.github.com/webhooks/ For rpminspect and pyparted, I use webhooks to trigger Travis-CI runs. For rpminspect, I trigger Copr builds on pushes so that new builds are continually available. You can select different events for the webhook, such as a release being made. I have not looked, but I hope there is something in the webhook API where you can query if Travis-CI is passing for that release. Alternatively, the webhook could just run a new Travis-CI job and if that passes, do the packit run.

There are webhooks for the check API, but afaik no one is using that interface - every CI system is setting commit statuses.

Not sure if commit comments webhook receives also the status updates.

I am not an expert on these webhooks, but it seems to me that we could be making use of the results. If I push to my repo on github, my Travis-CI jobs automatically run. I have it configured to run on each push and tag.

What I would like to have is:

1. Make a release by pushing a tag.
2. Travis-CI runs on that tag.
3. If that passes on the tag, a hook tells Packit to start building the packages in dist-git (ideally in the same way I am doing 'make koji' now)

With the Packit Service, it seems like there is a way to have it listen for when to build and that could be sent over by a webhook from github.