search

zerebubuth / openstreetmap-cgimap

A C++ implementation of the OpenStreetMap API map call.
http://wiki.openstreetmap.org/wiki/Cgimap
GNU General Public License v2.0
73 stars 38 forks source link

Build debian packages #449

Closed Firefishy closed 3 months ago

Firefishy commented 3 months ago

Build deb packages.

mmd-osm commented 3 months ago

Seems like there’s an issue with the pattern debian/artifacts/openstreetmap-cgimap-bin-_*.deb

The dash+underscore after bin doesn’t match any of these files: 

  Moved 'openstreetmap-cgimap-bin-dbgsym_0.9.3-1~jammy1+6e2be81~bookworm1_amd64.deb'
  Moved 'openstreetmap-cgimap-bin_0.9.3-1~jammy1+6e2be81~bookworm1_amd64.deb'
  Moved 'openstreetmap-cgimap_0.9.3-1~jammy1+6e2be81~bookworm1_amd64.buildinfo'
  Moved 'openstreetmap-cgimap_0.9.3-1~jammy1+6e2be81~bookworm1_amd64.changes'
mmd-osm commented 3 months ago

Looks better now!

tomhughes commented 3 months ago

Why does it have both jammy and bookworm in the version though?

mmd-osm commented 3 months ago

One of the two originates from debian/changelog, and we can remove it from there for the next release.

Firefishy commented 3 months ago

I will likely only be able to pick this up again on Monday. If anyone wants to work on it more before then feel free.

mmd-osm commented 3 months ago

I'm running some tests in https://github.com/mmd-osm/openstreetmap-cgimap/tree/test/449

https://github.com/mmd-osm/openstreetmap-cgimap/actions/runs/10438882672

Package files names look ok now: openstreetmap-cgimap-bin_2.0.0+e4e4c41~jammy1_amd64.deb, same for changelog:

openstreetmap-cgimap (2.0.0+e4e4c41~jammy1) jammy; urgency=medium

  * OAuth 1.0a removal (#354)
  * Basic auth removal (#362)
  * ...
mmd-osm commented 3 months ago

@Firefishy : I have added one commit to this PR now, which I previously tested in my test/449 branch. Please don't force push your local branch without fetching the latest updates from GitHub.

mmd-osm commented 3 months ago

Is there anything else we want to try out or are we good to merge this PR and release 2.0.0? What about arm64 support?

Firefishy commented 3 months ago

Is there anything else we want to try out or are we good to merge this PR and release 2.0.0? What about arm64 support?

Give me today, I will wrap it up and add "release" support to add all the deb files to the release eg like: https://github.com/StackExchange/dnscontrol/releases/tag/v4.12.5

mmd-osm commented 3 months ago

In another PR, I’ve mentioned the following repo where they’re already doing that: https://github.com/zerebubuth/openstreetmap-cgimap/pull/332#issuecomment-2288137018

Maybe some of it could be useful, not sure.

Direct link: https://github.com/trstringer/azblogfilter/blob/main/.github/workflows/release.yaml

mmd-osm commented 3 months ago

@Firefishy : I did some further testing and came up with https://github.com/mmd-osm/openstreetmap-cgimap/blob/test/449/.github/workflows/release.yml

This is how a release looks like when pushing a tag (I manually changed the status from draft to release, and ran the auto release note generation): https://github.com/mmd-osm/openstreetmap-cgimap/releases

According to the docs, the release process could be included in build.yml to avoid maintaining tons of different github action scripts.

      - name: Release
        uses: softprops/action-gh-release@v2
        if: startsWith(github.ref, 'refs/tags/')

-> going to try this out as well...

Release creation uses https://github.com/softprops/action-gh-release

mmd-osm commented 3 months ago

Now the release process is included in the Debian package build script.

https://github.com/mmd-osm/openstreetmap-cgimap/blob/test/449/.github/workflows/build.yml

Resulting 0.10.6 (test) release: https://github.com/mmd-osm/openstreetmap-cgimap/releases/tag/v0.10.6

This seems to work fine. We only need a single script for everything this way.

mmd-osm commented 3 months ago

I think I'm going to merge this now. Thanks a lot for your support @Firefishy !

mmd-osm commented 3 months ago

Added a wiki page to describe the new release process https://github.com/zerebubuth/openstreetmap-cgimap/wiki/Release-process-(rel-2.0.0-onwards)

mmd-osm commented 3 months ago

I published 2.0.0 earlier today: https://github.com/zerebubuth/openstreetmap-cgimap/releases/tag/v2.0.0

-> already uploaded to https://apt.openstreetmap.org/pool/main/o/openstreetmap-cgimap/ :tada:

mmd-osm commented 3 months ago

As mentioned in the release notes, some command line parameters are also gone now and need to be removed (!)

https://github.com/openstreetmap/chef/blob/9d12059127c0c5e6c6b1a88a4d2e95e21f1a03c7/cookbooks/web/recipes/cgimap.rb#L66-L70

No longer valid for 2.0.0 are both "--basic_auth_support false" and "--oauth_10_support false"

tomhughes commented 3 months ago

Yes I actually imported them into apt.openstreetmap.org yesterday but I didn't have time to upgrade things...

It would probably be a good idea for openstreetmap-cgimap to explicitly indicate that it replaces openstreetmap-cgimap-bin as currently it doesn't automatically upgrade and I'm not sure you can easily do anything other than a remove followed by an install.

mmd-osm commented 3 months ago

Right, I've also noticed that when doing some tests locally. https://www.debian.org/doc/debian-policy/ch-relationships.html#overwriting-files-and-replacing-packages-replaces describes some options to explicitly replace outdated packages. If someone has a good idea how exactly this works, I can include it in a 2.0.1 release.

Woazboat commented 3 months ago

I think adding this to the package in the debian/control file should work (didn't test it yet though)

Conflicts: openstreetmap-cgimap-bin
Replaces: openstreetmap-cgimap-bin

Replaces only does something when there's a conflict, so the Conflicts is also required

mmd-osm commented 3 months ago

Thanks! I'll give it a try in my repo clone and report back.

mmd-osm commented 3 months ago

I think this looks ok now:

sudo dpkg -i openstreetmap-cgimap_2.0.1+c20c5cd.jammy1_amd64.deb
Selecting previously unselected package openstreetmap-cgimap.
dpkg: considering removing openstreetmap-cgimap-bin in favour of openstreetmap-cgimap ...
dpkg: yes, will remove openstreetmap-cgimap-bin in favour of openstreetmap-cgimap
(Reading database ... 1731656 files and directories currently installed.)
Preparing to unpack openstreetmap-cgimap_2.0.1+c20c5cd.jammy1_amd64.deb ...
Unpacking openstreetmap-cgimap (2.0.1+c20c5cd~jammy1) ...
Removing openstreetmap-cgimap-bin (0.9.3-1~jammy1), to allow configuration of openstreetmap-cgimap (2.0.1+c20c5cd~jammy1) ...
Setting up openstreetmap-cgimap (2.0.1+c20c5cd~jammy1) ...
Processing triggers for man-db (2.10.2-1) ...

I've created a new PR #452 with the required changes.

mmd-osm commented 3 months ago

Release 2.0.1 is published now

mmd-osm commented 3 months ago

I hope we don't need ARM64 builds in the near future. It's incredibly slow due to the QEMU emulation (>1h for one package).

Firefishy commented 3 months ago

I hope we don't need ARM64 builds in the near future. It's incredibly slow due to the QEMU emulation (>1h for one package).

GitHub is expected to introduce public ARM64 runners late this year or early next year. Until then I have a cluster of Raspberry PI 5's which I plan to use as private runners, which OpenStreetMap projects are welcome to use. I will get them ready and let you know.

mmd-osm commented 3 months ago

My Raspberry PI 4 took around 33 minutes to build the package (with 3 parallel processes). I think that's quite ok given the fairly limited hw specs. A while ago, I've read about some issues w/ Debian on RPI 5 (also mentioned here: https://wiki.debian.org/RaspberryPiImages), but haven't really followed up on any updates. Is this still a thing?

Firefishy commented 3 months ago

A while ago, I've read about some issues w/ Debian on RPI 5 (also mentioned here: https://wiki.debian.org/RaspberryPiImages), but haven't really followed up on any updates. Is this still a thing?

I use Raspberry Pi OS's which is Debian, but with a few extra packages, minor customisations and repos. It is Debian 12 under the hood. It is possible to also run "pure" Debian 12 on a RPI, as seen by above link. Packages created with Raspberry Pi OS should have no issue running on Debian 12 arm64.