freedomofpress / dangerzone

Take potentially dangerous PDFs, office documents, or images and convert them to safe PDFs
https://dangerzone.rocks/
GNU Affero General Public License v3.0
3.58k stars 168 forks source link

Revert "ci: Disable Debian Trixie builds" #900

Closed legoktm closed 1 month ago

legoktm commented 1 month ago

This reverts commit 162ded6a7572dd542d4b3216d449daf65377deb4.

stdeb is back in unstable and trixie now (following https://tracker.debian.org/news/1553236/accepted-stdeb-0100-21-source-into-unstable/).

Refs #773.

legoktm commented 1 month ago

Hm, CI is still failing on trixie though, just for another reason now:

...
dpkg-buildpackage: info: source package dangerzone
dpkg-buildpackage: info: source version 0.7.0-1
dpkg-buildpackage: info: source distribution bionic
dpkg-buildpackage: info: source changed by Freedom of the Press Foundation <info@freedom.press>
dpkg-buildpackage: info: host architecture amd64
dpkg-buildpackage: warning: using a gain-root-command while being root
 dpkg-source --before-build .
 fakeroot debian/rules clean

Too long with no output (exceeded 10m0s): context deadline exceeded

I wonder why it's stalling there.

almet commented 1 month ago

I've tried to build the debian package locally from a trixie machine and… it works:

# 1. Create a trixie build env
./dev_scripts/env.py --distro debian --version trixie build-dev

# 2. Build the debian packages
./dev_scripts/env.py --distro debian --version trixie run --dev bash -c "cd dangerzone && ./install/linux/build-deb.py"

Spawning the CircleCI runners one more time to see if it was intermittent.

almet commented 1 month ago

Trying to understand what's going on, a process is being stuck when running fakeroot debian/rules clean, which is in turn running this debian/rules:

`%:
    dh $@ --with python3 --buildsystem=pybuild

When interrupting the process manually, I get this stacktrace:

CTraceback (most recent call last):
  File "/root/project/./install/linux/build-deb.py", line 80, in <module>
    main()
  File "/root/project/./install/linux/build-deb.py", line 63, in main
    run(
  File "/root/project/./install/linux/build-deb.py", line 22, in run
    subprocess.run(cmd, cwd=root, check=True)
  File "/usr/lib/python3.12/subprocess.py", line 550, in run
    stdout, stderr = process.communicate(input, timeout=timeout)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/subprocess.py", line 1201, in communicate
    self.wait()
  File "/usr/lib/python3.12/subprocess.py", line 1264, in wait
    return self._wait(timeout=timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/subprocess.py", line 2053, in _wait
    (pid, sts) = self._try_wait(0)
                 ^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/subprocess.py", line 2011, in _try_wait
    (pid, sts) = os.waitpid(self.pid, wait_flags)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
KeyboardInterrupt

Meaning that the python3 setup.py --command-packages=stdeb.command sdist_dsc bdist_deb is actually the one stalled.

almet commented 1 month ago

And, that's actually the other way around: the python3 setup.py --command-packages=stdeb.command sdist_dsc bdist_deb is what runs the other commands. (I'm not familiar with creating debian packages, as one can see, so this is learning the hard way :p)

I've been trying to strace the process to see what it's doing when stalled, but it's not authorized on circleCI unfortunately:

cat /proc/sys/kernel/yama/ptrace_scope 
1

(and the FS is read-only).


After some tinkering, I found some fakeroot zombies:

ps aux | grep 'Z'
root        5112  0.0  0.0      0     0 pts/2    Z    17:03   0:00 [fakeroot] <defunct>
root        5374  0.0  0.0      0     0 pts/4    Z+   17:36   0:00 [fakeroot] <defunct>
root        5504  0.0  0.0      0     0 pts/5    Z    18:23   0:00 [fakeroot] <defunct>
root        5577  0.0  0.0      0     0 pts/5    Z    18:33   0:00 [fakeroot] <defunct>

Attempting to run another fakeroot debian/rules clean command will result in a new zombie. So apparently something is going-on here.

legoktm commented 1 month ago

There were some changes to fakeroot over the past week that just landed in trixie, it's possible they were buggy. But really we should stop using fakeroot and set Rules-Requires-Root: no in our packaging, which is probably best done by dropping stdeb aka #773 :)

legoktm commented 1 month ago

Also I'm not sure if anyone else has actively used stdeb on trixie, it's possible other things were broken but don't fail at import time like the configparser thing.

almet commented 1 month ago

Thanks for the pointers. I'll take this as a chance to go in the right direction and change the way we build our Debian packages.

With that being said, It's worth noting that stdeb works on trixie on a local container (with no restrictions).

The problem might be related to the combination of CircleCI specific configuration of their worker machines and updated packages on trixie.

Because we're going away from CircleCI altogether (see #674), I will not commit more time to fixing the configuration there.

almet commented 1 month ago

Closing this in favor of #901. Thanks Kunal!