Closed fidergo-stephane-gourichon closed 4 years ago
Obtained a package for Ubuntu 18.04 with command line below.
Get out of the quicktile
directory. The package will be generated there, not inside the git-controlled directory so that they don't clash with the "git tree clean" check.
docker run --rm -it -v $PWD:/up ubuntu:bionic bash -c 'set -euxv ; \
apt-get update ; apt-get install -y --no-install-recommends git \
build-essential devscripts debhelper fakeroot \
dh-python python3 python3-pip \
python3-setuptools python3-gi python3-xlib python3-dbus \
gir1.2-gtk-3.0 gir1.2-wnck-3.0 ; \
bash /up/quicktile/recompile_local_debian_package.sh'
Packages get written into a compiled_packages/distro_version/packagename*
tree.
Thanks. I'm a little uncomfortable blindly accepting something I know so little about, but I do plan to accept this. Could you link me to the resources that taught you to do this?
and even some error, like missing man page
For 0.4.1 or 0.4.2, I was planning to switch from ePyDoc to Sphinx to get rid of the last (optional) dependency on Python 2 and enable major refactoring to use language constructs ePyDoc doesn't understand (mainly non-comment type annotations).
At that point, I'll either have Sphinx generate a man page or add help2man as a build-time dependency.
That said, a quick skim through what you're proposing to add doesn't show anything that's likely to need to change except the dependency on python3-dbus
which I'm planning to replace with a dependency on gir1.2-glib-2.0
accessed via python3-gi
.
Thanks. I'm a little uncomfortable blindly accepting something I know so little about, but I do plan to accept this. Could you link me to the resources that taught you to do this?
Sure. First, I have years of experience in working out things cleanly and simply when people are lost in a mess. It's part of my added value as an independent hands-on software consultant. I did if for a number of clients small and big.
Debian packaging looks like a moving target where lot of efforts have been invested in many directions. I don't claim to know it well, just figured out a configuration picking the patterns observed in existing software, with reuse and simplicity in mind, tested until it works, and can be refined.
The base spirit as I understand it is sound. Debian's goal is to allow to package a lot of programs with as few boilerplate as possible. With the years they included a lot of heuristics in the tools so that most package that follow a known pattern are easy to compile (autotools, cmake, python modules, etc). Also, tools help making sure every debian-specific changes are cleanly separated from the pristine upstream package. To avoid a mess, tools include many sanity checks, but since practices change a lot it's hard to find up-to-date and consistent information.
Now, the references:
debian/*
files, I got by using cme edit dpkg
mentioned in Packaging/Intro - Debian Wiki.debuild
did not know what to do. I did apt-get source
on a number of existing packages, read their debian/rules
to figure out how then handled python and /etc/xdg/autostart
, including packages backintime-1.1.24
, blueman-2.0.8
, caffeine-2.9.4
, displaycal-3.8.5.0
, python-xlib-0.23
, quicktile-0.4.0gtk3
, xcffib-0.8.1
. I got a list of relevant packages to look into, using the line below:
apt-file find /etc/xdg/autostart/ | cut -f 1 -d : | while read a ; do apt-file list $a | egrep -i '([^o]py)' ; done
Those hinted me about the main rule in debian/rules
https://github.com/fidergo-stephane-gourichon/quicktile/blob/b10c4b5b636ac98952e293e99d7812013a1859b3/debian/rules#L21
grep -rI buildsystem=pybuild */debian/rules
caffeine-2.9.4/debian/rules: dh $@ --with python3,bash-completion --buildsystem=pybuild
python-xlib-0.23/debian/rules: dh $@ --with python2,python3 --buildsystem=pybuild
quicktile-0.4.0gtk3/debian/rules: dh $@ --with python3 --buildsystem=pybuild
xcffib-0.8.1/debian/rules: dh $@ --with python2,python3 --buildsystem=pybuild
quicktile.desktop
. Finally, A short way to a deb package · Artem Sidorenko hinted me to go for the good old /usr/bin/install
.The debian/quicktile
mention in line https://github.com/fidergo-stephane-gourichon/quicktile/blob/b10c4b5b636ac98952e293e99d7812013a1859b3/debian/rules#L25 is not totally conformant to the DRY principle. There might be a cleaner way, like a makefile variable, although in practice it just works and shouldn't break unless the package gets renamed or Debian changes the build system.
and even some error, like missing man page
For 0.4.1 or 0.4.2, I was planning to switch from ePyDoc to Sphinx to get rid of the last (optional) dependency on Python 2 and enable major refactoring to use language constructs ePyDoc doesn't understand (mainly non-comment type annotations).
At that point, I'll either have Sphinx generate a man page or add help2man as a build-time dependency.
Good!
That said, a quick skim through what you're proposing to add doesn't show anything that's likely to need to change except the dependency on
python3-dbus
which I'm planning to replace with a dependency ongir1.2-glib-2.0
accessed viapython3-gi
.
Thanks for the feedback.
The first person that will benefit from the package is my 15-year old son. After superswitcher he'll have quicktile on his laptop. Thanks to you for doing the hard work.
Maybe we should consider installing quicktile.desktop
to /etc/xdg/autostart/
with something more pythonic, rather than putting an override in debian/control
. As far as I understand, the ideal Debian package just applies well-known rules chosen automatically based on the files present in the source tree. Another hint is that installing a .deskop
file in /etc/xdg/
is not Debian-specific, so should not be handled specially. Yet my feeling is that this is good enough for now. At least, it works for me and seems to balance simplicity in the mix.
Anyway, thanks again.
To avoid a mess, tools include many sanity checks, but since practices change a lot it's hard to find up-to-date and consistent information.
In other words, it's sort of like the time I asked my professor about researching back in university and got an answer that boiled down to "No, you're not missing anything and you've basically got it down. Our systems for sharing research with each other really are an appallingly inefficient mess that boils down to 'fight your way to knowing the correct jargon and then Google it'."
Thanks for confirming that I'm not just missing something and giving me a more directed way to get started. I'll take a look at it as soon as I've gotten some more urgent stuff out of the way, so it may be a week or two.
Maybe we should consider installing quicktile.desktop to /etc/xdg/autostart/ with something more pythonic, rather than putting an override in debian/control. As far as I understand, the ideal Debian package just applies well-known rules chosen automatically based on the files present in the source tree. Another hint is that installing a .deskop file in /etc/xdg/ is not Debian-specific, so should not be handled specially. There's probably a way, perhaps a Debian-independent makefile that Debian tools would call automatically, like make install?
That's another thing I've been meaning to get back to for a while. install.sh
was a hack to work around my not seeing a way to install to /etc
from setup.py
and not wanting to add a dependency on GNU Make for just that one thing.
In other words, it's sort of like the time I asked my professor about researching back in university and got an answer that boiled down to "No, you're not missing anything and you've basically got it down. Our systems for sharing research with each other really are an appallingly inefficient mess that boils down to 'fight your way to knowing the correct jargon and then Google it'."
Now it gets philosophical. The world is just as it is. Wishing it was better can be helpful or not, depending on the context. If you can install a "on-off switch" on those kind of wishes, then suddenly you get better when those wishes were a handicap.
Anyway, I have updated my branch to adjust the script that generates packages. These are still local packages, but at least they are gathered by distro, with a *.gitversion
tile that allows checking from which git revision they were generated.
compiled_packages/
compiled_packages/ubuntu-18.04
compiled_packages/ubuntu-18.04/quicktile_0.4.0gtk3
compiled_packages/ubuntu-18.04/quicktile_0.4.0gtk3/quicktile_0.4.0gtk3-1_amd64.build
compiled_packages/ubuntu-18.04/quicktile_0.4.0gtk3/quicktile_0.4.0gtk3-1.debian.tar.xz
compiled_packages/ubuntu-18.04/quicktile_0.4.0gtk3/quicktile_0.4.0gtk3-1_all.deb
compiled_packages/ubuntu-18.04/quicktile_0.4.0gtk3/quicktile_0.4.0gtk3.orig.tar.gz
compiled_packages/ubuntu-18.04/quicktile_0.4.0gtk3/quicktile_0.4.0gtk3-1_amd64.buildinfo
compiled_packages/ubuntu-18.04/quicktile_0.4.0gtk3/quicktile_0.4.0gtk3.gitversion
compiled_packages/ubuntu-18.04/quicktile_0.4.0gtk3/quicktile_0.4.0gtk3-1.dsc
compiled_packages/ubuntu-18.04/quicktile_0.4.0gtk3/quicktile_0.4.0gtk3-1_amd64.changes
compiled_packages/debian-10
compiled_packages/debian-10/quicktile_0.4.0gtk3
compiled_packages/debian-10/quicktile_0.4.0gtk3/quicktile_0.4.0gtk3-1_amd64.build
compiled_packages/debian-10/quicktile_0.4.0gtk3/quicktile_0.4.0gtk3-1.debian.tar.xz
compiled_packages/debian-10/quicktile_0.4.0gtk3/quicktile_0.4.0gtk3-1_all.deb
compiled_packages/debian-10/quicktile_0.4.0gtk3/quicktile_0.4.0gtk3.orig.tar.gz
compiled_packages/debian-10/quicktile_0.4.0gtk3/quicktile_0.4.0gtk3-1_amd64.buildinfo
compiled_packages/debian-10/quicktile_0.4.0gtk3/quicktile_0.4.0gtk3.gitversion
compiled_packages/debian-10/quicktile_0.4.0gtk3/quicktile_0.4.0gtk3-1.dsc
compiled_packages/debian-10/quicktile_0.4.0gtk3/quicktile_0.4.0gtk3-1_amd64.changes
compiled_packages/ubuntu-19.10
compiled_packages/ubuntu-19.10/quicktile_0.4.0gtk3
compiled_packages/ubuntu-19.10/quicktile_0.4.0gtk3/quicktile_0.4.0gtk3-1_amd64.build
compiled_packages/ubuntu-19.10/quicktile_0.4.0gtk3/quicktile_0.4.0gtk3-1.debian.tar.xz
compiled_packages/ubuntu-19.10/quicktile_0.4.0gtk3/quicktile_0.4.0gtk3-1_all.deb
compiled_packages/ubuntu-19.10/quicktile_0.4.0gtk3/quicktile_0.4.0gtk3.orig.tar.gz
compiled_packages/ubuntu-19.10/quicktile_0.4.0gtk3/quicktile_0.4.0gtk3-1_amd64.buildinfo
compiled_packages/ubuntu-19.10/quicktile_0.4.0gtk3/quicktile_0.4.0gtk3.gitversion
compiled_packages/ubuntu-19.10/quicktile_0.4.0gtk3/quicktile_0.4.0gtk3-1.dsc
compiled_packages/ubuntu-19.10/quicktile_0.4.0gtk3/quicktile_0.4.0gtk3-1_amd64.changes
Now it gets philosophical. The world is just as it is. Wishing it was better can be helpful or not, depending on the context. If you can install a "on-off switch" on those kind of wishes, then suddenly you get better when those wishes were a handicap.
It's more that I was caught in the grips of "They're professionals. They do this all the time. It can't possibly be this un-optimized. I must be missing something."
@fidergo-stephane-gourichon this is brillant, anyway you could upload the .deb file somewhere public? :)
Thanks @kbsali for the feedback. Will try, tonight. @ssokolow what do you think of a "github action" that would generate packages? I can prepare one on my fork then share with you via a PR. It would be not as good as a PPA or real integration in Debian, but it can be an intermediate step. Please tell me.
@fidergo-stephane-gourichon thanks a lot! I managed to compile the deb file with your docker command (you are just missing a little space here fakeroot\
).
Also echo "generated from git commit $GITREV" >"${OUTDIR}/${NAMEFORTAR}.gitversion"
in your recompile_local_debian_package.sh
should be move after mkdir -p "$OUTDIR"
(sorry, being lazy, not forking/fixing etc..) :)
Thanks @kbsali for reporting. Regarding fakeroot I corrected the comment above. Regarding mkdir and echo I adjusted the order.
@fidergo-stephane-gourichon My main concern with a GitHub action is the risk that either I or people using it will grow too comfortable with it and it'll hold back the creation of a proper PPA.
Firstly, thanks @ssokolow for this Christmas present! Thanks @fidergo-stephane-gourichon for the work to make it into a Debian package.
I've found I need to manually install python3-xlib
as a dependency to get the package to work. I used the docker command to build it, and it didn't add any dependencies to the deb file for some reason. I think more manual dependencies might be needed in debian/control
$ dpkg -I quicktile_0.4.0gtk3-1_all.deb
new Debian package, version 2.0.
size 88620 bytes: control archive=1712 bytes.
37 bytes, 1 lines conffiles
525 bytes, 14 lines control
3592 bytes, 39 lines md5sums
Package: quicktile
Version: 0.4.0gtk3-1
Architecture: all
Maintainer: Stéphane Gourichon <stephane_dpkg@gourichon.org>
Installed-Size: 269
Section: misc
Priority: optional
...
Other than that, it appears to be working for me under Ubuntu 18.04
Also @fidergo-stephane-gourichon if you build "artifacts" with a github action, then you're only a few steps away from making it a PPA. The generated debian files (.dsc, .debian.tar.xz, orig.tar.gz, .changes
) just need to be pushed to the PPA build (dput ppa:your-lp-id/ppa <source.changes>
). It's good to verify a package before you push it to the PPA anyway, so automating the building of the artifacts removes one more step from us mortals, and then just requires someone to dput the files after verifying they work correctly.
Firstly, thanks @ssokolow for this Christmas present!
I had actually hoped to have it out in time for Christmas, but said New Years because I suspected (correctly) that I'd get bogged down with unexpected delays.
Oh, it just occurred to me that you have python3-pip
in there unnecessarily. It should only be necessary for the installation options which use pip3
as a cross-distro alternative to apt
for keeping track of what to uninstall.
Also @fidergo-stephane-gourichon if you build "artifacts" with a github action, then you're only a few steps away from making it a PPA. The generated debian files (.dsc, .debian.tar.xz, orig.tar.gz, .changes) just need to be pushed to the PPA build (dput ppa:your-lp-id/ppa
). It's good to verify a package before you push it to the PPA anyway, so automating the building of the artifacts removes one more step from us mortals, and then just requires someone to dput the files after verifying they work correctly.
Sounds like a great thing to automate once I finish reviving ITAD Importer (and possibly finishing whipping up Minimum Viable Products for certain helper utilties I need) and come back to this to write the functional test harness.
At the moment, test_functional.py
isn't very useful because it doesn't assert anything beyond "program doesn't die" and, because the harness doesn't yet generate any mock X11 windows to move around and most of the commands are flagged as operating on an active window, they bail out in the "get active window code" that runs before anything command-specific.
However, once that's done, my plan is to have the harness iterate through a bunch of different WMs and, if necessary, different desktop configurations, spinning up an Xvfb or Xephyr instance for each one and then performing QuickTile operations and checking for correct results. That should give me the confidence necessary to even offer Continuous Deployment to a PPA from HEAD if I so choose.
Just created #107, another regression introduced by branch gtk3_port.
With luck, I should have an 0.4.0 release candidate ready by the end of tomorrow that this can be rebased and merged onto.
Rebased on top of current gtk3_port
( commit bce6d7f Add more unit tests for issue 45 ).
Running locally works.
Building package succeeds but fails to start:
/usr/bin/quicktile --daemonize
Traceback (most recent call last):
File "/usr/bin/quicktile", line 6, in <module>
from pkg_resources import load_entry_point
File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 3250, in <module>
@_call_aside
File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 3234, in _call_aside
f(*args, **kwargs)
File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 3263, in _initialize_master_working_set
working_set = WorkingSet._build_master()
File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 583, in _build_master
ws.require(__requires__)
File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 900, in require
needed = self.resolve(parse_requirements(requirements))
File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 786, in resolve
raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'QuickTile==0.4' distribution was not found and is required by the application
That's usually a bug in setuptools that causes it to fail to plumb together the /usr/local/bin/quicktile
script and the /usr/local/lib/python*/dist-packages/*/quicktile
module.
See if python3 -m quicktile
works.
See if
python3 -m quicktile
works.
That does not complain. And this works:
python3 -m quicktile --daemonize
Is a fix is to be done your side?
I never understood why it happens in the first place. From my perspective, setuptools looks like a confusing, over-engineered mess for 99% of use-cases, but setup.py
is too declarative to install a script into the user's PATH
any other way without reinventing a ton of machinery.
The advice I always gave was to find
all quicktile stuff inside /usr
, delete them, try reinstalling, and hope that fixes it.
Failing that, the generated /usr/bin/quicktile
will contain something like this...
#!/usr/bin/python3
# EASY-INSTALL-ENTRY-SCRIPT: 'QuickTile==0.4','console_scripts','quicktile'
__requires__ = 'QuickTile==0.4'
import re
import sys
from pkg_resources import load_entry_point
if __name__ == '__main__':
sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0])
sys.exit(
load_entry_point('QuickTile==0.4', 'console_scripts', 'quicktile')()
)
...and replacing it with this should make it work:
#!/usr/bin/python3
from quicktile.__main__ import main
main()
Hhm, could not reproduce. The actual cause might be that I mistakenly installed the package for 18.04 instead of the 19.10 I'm using.
Anyway, there were nothing related to quicktile in /usr because I tend to never to things like sudo make install
.
find /usr/ -iname "*quicktile*"
/usr/share/doc/quicktile
/usr/share/doc/quicktile/quicktile
/usr/share/applications/quicktile.desktop
/usr/lib/python3.6/dist-packages/QuickTile-0.4.egg-info
/usr/lib/python3.6/dist-packages/quicktile
/usr/bin/quicktile
# dpkg --purge quicktile
(Lecture de la base de données... 447762 fichiers et répertoires déjà installés.)
Suppression de quicktile (0.4.0gtk3-1) ...
Purge des fichiers de configuration de quicktile (0.4.0gtk3-1) ...
Traitement des actions différées (« triggers ») pour gnome-menus (3.32.0-1ubuntu1) ...
Traitement des actions différées (« triggers ») pour desktop-file-utils (0.24-1ubuntu1) ...
Traitement des actions différées (« triggers ») pour mime-support (3.63ubuntu1) ...
# find /usr/ -iname "*quicktile*"
(no output)
Anyway, package work now. Ok to rebase this branch onto your merged master.
master
is now up to date and I'll be removing the gtk3_port
branch momentarily.
There. Merged and tagged as the final commits of v0.4.0
.
Next goal on this topic: Integrating it into the testing routine and Travis-CI builds to make sure it doesn't break.
Hmm. I just noticed something from opening the generated .deb
up with file-roller that I missed before.
It looks like it's mistaking source files for documentation and making copies to be installed to /usr/share/doc/quicktile/quicktile/
, some gzipped and some not.
Indeed. This is the result of the content of debian/docs
.
What is supposed to go into /usr/share/doc/quicktile/
? The raw content of the docs subdirectory? Something else?
Looks like the latest commit plus these hints go towards a correct way:
pip3 install sphinx_autodoc_typehints sphinxcontrib.autoprogram
make html
sphinx-build -b html -d _build/doctrees . _build/html
Running Sphinx v2.4.0
loading translations [en]... done
making output directory... done
loading intersphinx inventory from https://pycairo.readthedocs.io/en/latest/objects.inv...
loading intersphinx inventory from https://dbus.freedesktop.org/doc/dbus-python/objects.inv...
loading intersphinx inventory from https://lazka.github.io/pgi-docs/Gtk-3.0/objects.inv...
loading intersphinx inventory from https://lazka.github.io/pgi-docs/Gdk-3.0/objects.inv...
loading intersphinx inventory from https://lazka.github.io/pgi-docs/GdkX11-3.0/objects.inv...
loading intersphinx inventory from https://lazka.github.io/pgi-docs/GLib-2.0/objects.inv...
loading intersphinx inventory from https://docs.python.org/3/objects.inv...
loading intersphinx inventory from https://www.sphinx-doc.org/en/master/objects.inv...
loading intersphinx inventory from https://lazka.github.io/pgi-docs/Wnck-3.0/objects.inv...
building [mo]: targets for 0 po files that are out of date
building [html]: targets for 22 source files that are out of date
updating environment: [new config] 22 added, 0 changed, 0 removed
reading sources... [100%] usage
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... WARNING: unsupported theme option 'badge_branch' given
WARNING: unsupported theme option 'canonical_url' given
done
writing output... [100%] usage
generating indices... genindex py-modindexdone
highlighting module code... [100%] test_quicktile
writing additional pages... searchdone
copying images... [100%] wrench.png
copying static files... ... done
copying extra files... done
dumping search index in English (code: en)... done
dumping object inventory... done
build succeeded, 2 warnings.
The HTML pages are in _build/html.
Build finished. The HTML pages are in _build/html.
I'll need to look into Debian policy on what kind of docs are and aren't supposed to go into /usr/share/doc/
, but a copy of the source code most definitely is not what goes in there.
A quick look at some of the packages in my own /usr/share/doc/
suggests README.rst
and, optionally, a rendered copy of the docs. (Something like running cd docs; make html
and taking docs/_build/html
as in /usr/share/doc/tk-tile/html
or, if I can figure out what's tripping up LaTeX, a PDF from make latexpdf
as in /usr/share/doc/valgrind/valgrind_manual.pdf.gz
)
Sphinx, sphinx_autodoc_typehints, and sphinxcontrib.autoprogram will also let you build and preview the beginnings of a manpage using
(cd docs; make man)
man -l docs/_build/man/quicktile.1
I've pushed another commit towards including the correct doc.
Package now includes html and man page! Wow, you've written clear doc!
Here are some warning I noticed during docker build:
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... WARNING: unsupported theme option 'badge_branch' given
WARNING: unsupported theme option 'canonical_url' given
done
I've pushed another commit towards including the correct doc.
Package now includes html and man page!
You'll have to open a new PR. GitHub won't let me re-merge this one.
Wow, you've written clear doc!
I have perfectionist tendencies. That's what happens when I get into my doc-writing. (I'm actually still not satisfied with parts of it.)
Here are some warning I noticed during docker build:
I think you might be running an older version of the Alabaster theme than I am.
Both of those theme options are listed at https://alabaster.readthedocs.io/en/latest/customization.html and neither of those warnings appear in either my local build or the Travis-CI build which produces the live site.
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
writing output... [100%] usage
generating indices... genindex py-modindexdone
highlighting module code... [100%] test_quicktile
writing additional pages... searchdone
copying images... [100%] diagrams/png/monitor-next-all.png
copying static files... ... done
copying extra files... done
dumping search index in English (code: en)... done
dumping object inventory... done
build succeeded.
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
writing output... [ 4%] apidocs/__main__
writing output... [ 9%] apidocs/commands
writing output... [ 13%] apidocs/dbus_api
writing output... [ 18%] apidocs/functional_harness/env_general
writing output... [ 22%] apidocs/functional_harness/x_server
writing output... [ 27%] apidocs/gtkexcepthook
writing output... [ 31%] apidocs/index
writing output... [ 36%] apidocs/keybinder
writing output... [ 40%] apidocs/layout
writing output... [ 45%] apidocs/test_functional
writing output... [ 50%] apidocs/test_quicktile
writing output... [ 54%] apidocs/tests
writing output... [ 59%] apidocs/util
writing output... [ 63%] apidocs/wm
writing output... [ 68%] cli
writing output... [ 72%] commands
writing output... [ 77%] config
writing output... [ 81%] developing
writing output... [ 86%] faq
writing output... [ 90%] index
writing output... [ 95%] installation
writing output... [100%] usage
This goes a good way to fix #105.
Thank you for your attention.