Debian / debiman

debiman generates a static manpage HTML repository out of a Debian archive
Apache License 2.0
195 stars 46 forks source link

Implement slave alternative manpages #12

Open stapelberg opened 7 years ago

stapelberg commented 7 years ago

A large number of packages makes manpages available via slave alternatives. See https://codesearch.debian.net/search?q=path%3Adebian%2F+update-alternatives+--install for an upper bound.

This is tricky to implement, because slave alternatives are configured in the postinst maintainer script, i.e. via update-alternatives calls in arbitrary shell script. It would be brittle to parse that shell script without running it, not to mention that its behavior might depend on the package actually being installed.

It seems like the only solution is to actually install the package and then introspect the alternatives database, e.g. /var/lib/dpkg/alternatives/vi.

A few test cases are:

stapelberg commented 7 years ago

One thing worth investigating is whether one could install packages in a Linux user namespace to reasonably well sandbox the installation.

stapelberg commented 7 years ago

I’ve filed a feature request against piuparts: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=850917

stapelberg commented 7 years ago

The patch is now live in piuparts and reflected in a number of (but not all) logfiles, see e.g. /srv/piuparts.debian.org/htdocs/sid/pass/vim-nox_2:8.0.0197-3.log:

LOG-ALTERNATIVES: dpkg=vim-nox: piuparts=vim-nox=2:8.0.0197-3: /usr/bin/update-alternatives --install /usr/bin/vi vi /usr/bin/vim.nox 40 --slave /usr/share/man/fr/man1/vi.1.gz vi.fr.1.gz /usr/share/man/fr/man1/vim.1.gz --slave /usr/share/man/it/man1/vi.1.gz vi.it.1.gz /usr/share/man/it/man1/vim.1.gz --slave /usr/share/man/pl/man1/vi.1.gz vi.pl.1.gz /usr/share/man/pl/man1/vim.1.gz --slave /usr/share/man/ru/man1/vi.1.gz vi.ru.1.gz /usr/share/man/ru/man1/vim.1.gz --slave /usr/share/man/ja/man1/vi.1.gz vi.ja.1.gz /usr/share/man/ja/man1/vim.1.gz --slave /usr/share/man/man1/vi.1.gz vi.1.gz /usr/share/man/man1/vim.1.gz

Next up: making Debiman read these logs and parse the command lines.

stapelberg commented 7 years ago

I implemented support and verified my changes by running debiman with and without the new -alternatives_dir flag, then generating rwmaps and diffing them:

debiman-idx2rwmap -index=before/auxserver.idx -output_dir=before/rwmap/
debiman-idx2rwmap -index=after/auxserver.idx -output_dir=after/rwmap/
(cd before/rwmap && LC_ALL=C sort * > output)
(cd after/rwmap && LC_ALL=C sort * > output)
diff --speed-large-files -u before/rwmap/output after/rwmap/output

The diff only contains expected additions, which verifies the change works as expected.

I’ll try cleaning up, pushing and deploying this tomorrow.

stapelberg commented 7 years ago

I made a mistake yesterday by only specifying -force_rerender, but what we needed was -force_reextract, as the links are created when extracting packages.

I’m currently doing another run with -force_reextract, which should finish today and then be distributed to the static mirrors. I’ll check back in tomorrow morning.

stapelberg commented 7 years ago

This is now live. I’ll still need to set up automated updates of the piuparts data files.

jeffmcjunkin commented 7 years ago

Should README.me be updated to reflect this issue being closed?

Currently, there is one known bug with regards to completeness (#12).

stapelberg commented 7 years ago

Thanks for the hint. The automated updates of the data files aren’t done yet. Let me reopen the bug until this is fixed, then also update the README.

stapelberg commented 7 years ago

Status update: automated updates are now in place.

https://piuparts.debian.org/for-manpages.d.o/ is still missing buster.json.gz (I’ve asked Holger to pull a corresponding change).

Once that file appears, I’ll push a change to update the README.

TobiX commented 6 years ago

Is this why psql is missing? https://manpages.debian.org/stretch/postgresql-client-9.6/index.html (Or is that a new bug?)

stapelberg commented 6 years ago

Is this why psql is missing? https://manpages.debian.org/stretch/postgresql-client-9.6/index.html (Or is that a new bug?)

Thanks for the report. psql(1) is missing because of a logic bug in that infrastructure. I filed https://github.com/Debian/debiman/issues/106 for that.