datalad / datalad-installer

Installation script for Datalad and related components
MIT License
6 stars 3 forks source link

add datalad/packages support for Debian-based #142

Closed yarikoptic closed 1 year ago

yarikoptic commented 1 year ago

to complement https://github.com/datalad/datalad-installer/pull/136/files which added for OSX (originally was only for windows) and should be as easy to add for ubuntu, thus hopefully avoiding need to specify GITHUB_TOKEN if to go through datalad/git-annex:release to resolve https://github.com/datalad/datalad-installer/issues/137 (as in https://github.com/DanielDent/git-annex-remote-rclone/pull/60 )

jwodder commented 1 year ago

@yarikoptic What exactly is the format of the filename of the .deb? The latest packages all seem to be of the form git-annex-standalone_{version}-1.ndall+1_amd64.deb, but the older ones (which all include Git details in the version) have names of the form git-annex-standalone_{version}-1~ndall+1_amd64.deb (with a tilde instead of a period before "ndall").

yarikoptic commented 1 year ago

hm, that is odd! I was expecting ~ndall too! apparently:

we do build as `~ndall` ```shell (git)smaug:/mnt/btrfs/datasets/datalad/ci/git-annex[master]releases/10.20220127 $> grep '\.ndall' git-annex-build.log $> grep '~ndall' git-annex-build.log + dpkg-source -x git-annex_10.20220127-1~ndall+1.dsc git-annex-source dpkg-source: warning: failed to verify signature on ./git-annex_10.20220127-1~ndall+1.dsc dpkg-source: info: unpacking git-annex_10.20220127-1~ndall+1.tar.gz dpkg-buildpackage: info: source version 10.20220127-1~ndall+1 dpkg-deb: building package 'git-annex-standalone' in '../git-annex-standalone_10.20220127-1~ndall+1_amd64.deb'. dpkg-genchanges --build=binary >../git-annex_10.20220127-1~ndall+1_amd64.changes signfile git-annex_10.20220127-1~ndall+1_amd64.buildinfo signfile git-annex_10.20220127-1~ndall+1_amd64.changes ```

but we have .ndall on https://github.com/datalad/git-annex/releases/tag/10.20221212 .

and it seems that we are uploading `~ndall` to github ```shell (git)smaug:/mnt/btrfs/datasets/datalad/ci/git-annex/builds/2022/01[master]git $> grep Uploaded manual-20220127/build-ubuntu.yaml-557-85dd9355-failed/1_build-package\ \(1\).txt 2022-01-27T20:17:59.1084634Z Uploaded /home/runner/work/git-annex/git-annex/git-annex_10.20220127-1~ndall+1.tar.gz (12.6%) bytes 0:8388607 2022-01-27T20:17:59.6922032Z Uploaded /home/runner/work/git-annex/git-annex/git-annex_10.20220127-1~ndall+1.tar.gz (25.3%) bytes 8388608:16777215 2022-01-27T20:18:00.2466394Z Uploaded /home/runner/work/git-annex/git-annex/git-annex_10.20220127-1~ndall+1.tar.gz (38.0%) bytes 16777216:25165823 2022-01-27T20:18:00.7218420Z Uploaded /home/runner/work/git-annex/git-annex/git-annex_10.20220127-1~ndall+1.tar.gz (50.7%) bytes 25165824:33554431 2022-01-27T20:18:01.1824318Z Uploaded /home/runner/work/git-annex/git-annex/git-annex_10.20220127-1~ndall+1.tar.gz (63.4%) bytes 33554432:41943039 2022-01-27T20:18:01.7425801Z Uploaded /home/runner/work/git-annex/git-annex/git-annex_10.20220127-1~ndall+1.tar.gz (76.1%) bytes 41943040:50331647 2022-01-27T20:18:01.7868165Z Uploaded /home/runner/work/git-annex/git-annex/git-annex-standalone_10.20220127-1~ndall+1_amd64.deb (16.9%) bytes 0:8388607 2022-01-27T20:18:02.1765197Z Uploaded /home/runner/work/git-annex/git-annex/git-annex-standalone_10.20220127-1~ndall+1_amd64.deb (33.8%) bytes 8388608:16777215 2022-01-27T20:18:02.3040176Z Uploaded /home/runner/work/git-annex/git-annex/git-annex_10.20220127-1~ndall+1.tar.gz (88.8%) bytes 50331648:58720255 2022-01-27T20:18:02.6531776Z Uploaded /home/runner/work/git-annex/git-annex/git-annex-standalone_10.20220127-1~ndall+1_amd64.deb (50.7%) bytes 16777216:25165823 2022-01-27T20:18:02.7661167Z Uploaded /home/runner/work/git-annex/git-annex/git-annex_10.20220127-1~ndall+1.tar.gz (100.0%) bytes 58720256:66064905 2022-01-27T20:18:03.0966682Z Uploaded /home/runner/work/git-annex/git-annex/git-annex-standalone_10.20220127-1~ndall+1_amd64.deb (67.6%) bytes 25165824:33554431 2022-01-27T20:18:03.5383764Z Uploaded /home/runner/work/git-annex/git-annex/git-annex-standalone_10.20220127-1~ndall+1_amd64.deb (84.5%) bytes 33554432:41943039 2022-01-27T20:18:04.0101998Z Uploaded /home/runner/work/git-annex/git-annex/git-annex-standalone_10.20220127-1~ndall+1_amd64.deb (100.0%) bytes 41943040:49631977 ```

so it must be github which replaces ~ with . in the filename. Could you please confirm that @jwodder and see if there is some way for it to retain ~ in the filename?

if there would be no way -- I will I guess adjust http://datasets.datalad.org/datalad/packages/update.sh to replace . with ~ as IMHO it should be.

Note: metadata about package version is within .deb , so even if user downloads https://github.com/datalad/git-annex/releases/download/10.20221212/git-annex-standalone_10.20221212-1.ndall+1_amd64.deb and installs it, dpkg/apt would know it as ~ndall.

jwodder commented 1 year ago

@yarikoptic I can confirm that GitHub changes tildes in release asset filenames to periods on upload. ~I can't find a way to prevent this, nor does this behavior seem to be documented anywhere.~ Specifically:

GitHub renames asset filenames that have special characters, non-alphanumeric characters, and leading or trailing periods.