Closed maltfield closed 1 year ago
Iterating on GitHub Action's shared runners is slow, so I'm first testing if I can achieve [3] above locally, using the following commands to initialize a new docker container running the latest version of Debian.
sudo apt-get install docker.io
sudo bash -c 'gpasswd -a "${SUDO_USER}" docker'
su - `whoami`
export DOCKER_CONTENT_TRUST=1
docker run --rm --cap-add "NET_ADMIN" -it --entrypoint /bin/bash debian:stable-slim
apt-get update
apt-get install git
git clone --depth=1 https://github.com/BusKill/buskill-app.git
cd buskill-app
build/linux/buildAppImage.sh
It's quite possible that I will need to tweak the docker run
command to give the container access to USB devices
That's killing my data. Here's a faster way to initialize it so that the git repo only has to be cloned once and shared between future docker instances
cd ${HOME}
git clone --depth=1 https://github.com/BusKill/buskill-app.git
sudo apt-get install docker.io
sudo bash -c 'gpasswd -a "${SUDO_USER}" docker'
su - `whoami`
cd ${HOME}/buskill-app
export DOCKER_CONTENT_TRUST=1
docker run --rm --cap-add "NET_ADMIN" -v "`pwd`:/root/buskill-app" -it --entrypoint /bin/bash debian:stable-slim
cd /root/buskill-app
build/linux/buildAppImage.sh
dist/buskill-lin-*/buskill-*.AppImage --arm
Actually, even faster (just to figure out the USB stuff in docker) is to just download the latest build. I can switch to building at the time the Unit Test runs later..
cd ${HOME}
wget https://github.com/BusKill/buskill-app/releases/download/v0.5.0/buskill-lin-v0.5.0-x86_64.tbz
tar -xjvf buskill-lin-v0.5.0-x86_64.tbz
sudo apt-get install docker.io
sudo bash -c 'gpasswd -a "${SUDO_USER}" docker'
su - `whoami`
cd ${HOME}/buskill-lin-v0.5.0-x86_64
export DOCKER_CONTENT_TRUST=1
docker run --rm --cap-add "NET_ADMIN" -v "`pwd`:/root/buskill-app" -it --entrypoint /bin/bash debian:stable-slim
cd /root/buskill-app
/buskill-v0.5.0.AppImage --arm
I gave up with (A).
With (B), I got an error about FUSE
root@4c2722dc995e:~/buskill-app# dist/buskill-lin-*/buskill-*.AppImage --arm
dlopen(): error loading libfuse.so.2
AppImages require FUSE to run.
You might still be able to extract the contents of this AppImage
if you run it with the --appimage-extract option.
See https://github.com/AppImage/AppImageKit/wiki/FUSE
for more information
I got the same error with the latest stable release in (C)
root@2354fd6a7716:~/buskill-app# ./buskill-v0.5.0.AppImage --arm
dlopen(): error loading libfuse.so.2
AppImages require FUSE to run.
You might still be able to extract the contents of this AppImage
if you run it with the --appimage-extract option.
See https://github.com/AppImage/AppImageKit/wiki/FUSE
for more information
root@2354fd6a7716:~/buskill-app#
I tried installing libfuse2, but I couldn't add it as a module
root@4c2722dc995e:~/buskill-app# apt-get install libfuse2
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Suggested packages:
fuse
The following NEW packages will be installed:
libfuse2
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 130 kB of archives.
After this operation, 362 kB of additional disk space will be used.
Get:1 http://deb.debian.org/debian stable/main amd64 libfuse2 amd64 2.9.9-5 [130 kB]
Fetched 130 kB in 2s (55.1 kB/s)
debconf: delaying package configuration, since apt-utils is not installed
Selecting previously unselected package libfuse2:amd64.
(Reading database ... 19778 files and directories currently installed.)
Preparing to unpack .../libfuse2_2.9.9-5_amd64.deb ...
Unpacking libfuse2:amd64 (2.9.9-5) ...
Setting up libfuse2:amd64 (2.9.9-5) ...
Processing triggers for libc-bin (2.31-13+deb11u3) ...
root@4c2722dc995e:~/buskill-app#
root@4c2722dc995e:~/buskill-app#
root@4c2722dc995e:~/buskill-app# dist/buskill-lin-*/buskill-*.AppImage --arm
fuse: device not found, try 'modprobe fuse' first
Cannot mount AppImage, please check your FUSE setup.
You might still be able to extract the contents of this AppImage
if you run it with the --appimage-extract option.
See https://github.com/AppImage/AppImageKit/wiki/FUSE
for more information
open dir error: No such file or directory
root@4c2722dc995e:~/buskill-app# modprobe fuse
bash: modprobe: command not found
root@4c2722dc995e:~/buskill-app#
TODO: figure out how to run the buskill app inside of a debian docker container without getting FUSE errors from the AppImage.
Looks like I encountered this issue running my app in docker before on 2020-05-31, which was the day I migrated cross-platform-python-gui
(the Hello World
predecessor that became this app's foundation) from GitLab to GitHub (after I discovered GitHub had free shared runners for MacOS but Gitlab did not)
N. I went ahead and migrated to github https://github.com/maltfield/cross-platform-python-gui/
O. I quickly hit an issue with `fuse` on the github action script for building the linux AppImage. From my experience with gitlab on this, the docker host has to have the fuse module enabled for its docker containers to be able to use it. In gitlab, just installing 'fuse' from apt was enough. But not so in github https://github.com/maltfield/cross-platform-python-gui/runs/725594617?check_suite_focus=true
2020-05-31T21:33:35.5478138Z + chmod +x /tmp/appimagetool.AppImage
2020-05-31T21:33:35.5487678Z + mkdir dist
2020-05-31T21:33:35.5517125Z + /tmp/appimagetool.AppImage /tmp/kivy_appdir dist/helloWorld.AppImage
2020-05-31T21:33:35.5522850Z fuse: device not found, try 'modprobe fuse' first
2020-05-31T21:33:35.7058092Z open dir error: No such file or directory
2020-05-31T21:33:35.7058725Z + uname -a
Q. While searching, I discovered this indicating that nested virtualization isn't enabled on any github runners in azure https://github.com/actions/virtual-environments/issues/183
R. but, interestingly, all MacOS runners support nested virtualization *and* have vagrant & virtualbox installed *shrug* https://github.com/actions/virtual-environments/issues/433#issuecomment-610789144
Looks like nested virtualization is not possible on the azure ds2 instance types used in GitHub Shared Runners
Ugh, the above ticket was marked as wontfix
and closed in April 2020.
I guess that blocks running these tests inside a docker container that I setup. I can use docker on GitHub Actions Shared Runners, but then I can't control the host of the docker container, which may prevent me from being able to simulate a USB drive ejection (as we need for BusKill).
Options I can think of:
docker
command to work on MacOSApparently MacOS can now be run inside a docker container on a Linux system? That may make switching back to GitLab a possible choice (GitLab has nested virtualization but no MacOS shared runners)
I also did some googling to see if it's now possible to launch a CI/CD pipeline on a free MacOS shared runner in GitLab.com. Here's the timeline I gathered from my research:
2018-10: First requested by GitLab users https://gitlab.com/gitlab-com/gl-infra/reliability/-/issues/5294 2020-10: GitLab starts closed beta testing for shared macOS runners https://gitlab.com/groups/gitlab-org/-/epics/3926 2021-08: GitLab starts open beta testing for shared macOS runners https://gitlab.com/gitlab-org/gitlab/-/issues/269756
(the future) 2022-11: GitLab is planning to make macOS shared runners part of the Limited Availability Fleet, so it will be more generally available but only for whitelisted customers (I read this as: it's still a closed beta and this will not change before next year) https://gitlab.com/gitlab-org/gitlab/-/issues/342848
I'm just going to leave this here https://github.com/kms15/inception
I finally got docker working on our MacMini. It appears that it's impossible to install docker without a GUI. So infuriatingly short-sighted.
I opened a bug to fix this here https://github.com/docker/for-mac/issues/6504
I downloaded the buskill release for linux onto our Mac Mini, launched a new Debian docker container in it, and hit the same issue with the AppImage running (lacking fuse).
cd ${HOME}
wget https://github.com/BusKill/buskill-app/releases/download/v0.5.0/buskill-lin-v0.5.0-x86_64.tbz
tar -xjvf buskill-lin-v0.5.0-x86_64.tbz
cd ${HOME}/buskill-lin-v0.5.0-x86_64
export DOCKER_CONTENT_TRUST=1
docker run --privileged --rm --cap-add "NET_ADMIN" -v "`pwd`:/root/buskill-app" -it --entrypoint /bin/bash debian:stable-slim
apt-get update
apt-get install libfuse2
cd /root/buskill-app
./buskill-v0.5.0.AppImage --arm
I couldn't run modprobe
root@c5c454565069:~/buskill-app# ./buskill-v0.5.0.AppImage --arm
fuse: device not found, try 'modprobe fuse' first
Cannot mount AppImage, please check your FUSE setup.
You might still be able to extract the contents of this AppImage
if you run it with the --appimage-extract option.
See https://github.com/AppImage/AppImageKit/wiki/FUSE
for more information
open dir error: No such file or directory
root@c5c454565069:~/buskill-app# modprobe fuse
bash: modprobe: command not found
root@c5c454565069:~/buskill-app#
I tried with the --privliged
argument. That appears to have resolved the fuse issues (even though I still don't have access to modprobe
or lsmod
).
But now I have a different error regarding libusb
maltfield@5129 buskill-lin-v0.5.0-x86_64 % docker run --privileged --rm --cap-add "NET_ADMIN" -v "`pwd`:/root/buskill-app" -it --entrypoint /bin/bash debian:stable-slim
root@75b34c9e3f49:/# apt-get update
Get:1 http://deb.debian.org/debian stable InRelease [116 kB]
Get:2 http://deb.debian.org/debian-security stable-security InRelease [48.4 kB]
Get:3 http://deb.debian.org/debian stable-updates InRelease [44.1 kB]
Get:4 http://deb.debian.org/debian stable/main amd64 Packages [8184 kB]
Get:5 http://deb.debian.org/debian-security stable-security/main amd64 Packages [186 kB]
Get:6 http://deb.debian.org/debian stable-updates/main amd64 Packages [6344 B]
Fetched 8585 kB in 2s (3633 kB/s)
Reading package lists... Done
root@75b34c9e3f49:/# apt-get install libfuse2
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Suggested packages:
fuse
The following NEW packages will be installed:
libfuse2
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 130 kB of archives.
After this operation, 362 kB of additional disk space will be used.
Get:1 http://deb.debian.org/debian stable/main amd64 libfuse2 amd64 2.9.9-5 [130 kB]
Fetched 130 kB in 0s (1944 kB/s)
debconf: delaying package configuration, since apt-utils is not installed
Selecting previously unselected package libfuse2:amd64.
(Reading database ... 6661 files and directories currently installed.)
Preparing to unpack .../libfuse2_2.9.9-5_amd64.deb ...
Unpacking libfuse2:amd64 (2.9.9-5) ...
Setting up libfuse2:amd64 (2.9.9-5) ...
Processing triggers for libc-bin (2.31-13+deb11u4) ...
root@75b34c9e3f49:/#
root@75b34c9e3f49:/# cd /root/buskill-app
root@75b34c9e3f49:~/buskill-app# ./buskill-v0.5.0.AppImage --arm
buskill version {'VERSION': 'v0.5.0', 'GITHUB_REF': 'refs/heads/v0.5.0', 'GITHUB_SHA': '996c0f6147eacc2999433fd1e187b198b1becdf6', 'SOURCE_DATE_EPOCH': '1658495593'}
DEBUG: EXE_PATH:|/root/buskill-app/buskill-v0.5.0.AppImage|
DEBUG: EXE_DIR:|/root/buskill-app|
DEBUG: EXE_FILE:|buskill-v0.5.0.AppImage|
DEBUG: APP_DIR:|/root/buskill-app|
DEBUG: APPS_DIR:|/root|
DEBUG: os.environ['PATH']:|/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/root/buskill-app:/root/buskill-app|
INFO: using DATA_DIR:|/root/.buskill|
DEBUG: attempting to arm BusKill via <bound method BusKill.armNix of <packages.buskill.BusKill object at 0x7f3f1ded6590>>()
INFO: BusKill is armed. Listening for removal event.
INFO: To disarm the CLI, exit with ^C or close this terminal
Process Process-1:
Traceback (most recent call last):
File "/tmp/.mount_buskilebOSKC/opt/python3.7/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
self.run()
File "/tmp/.mount_buskilebOSKC/opt/python3.7/lib/python3.7/multiprocessing/process.py", line 99, in run
self._target(*self._args, **self._kwargs)
File "/tmp/.mount_buskilebOSKC/opt/src/packages/buskill/__init__.py", line 624, in armNix
with usb1.USBContext() as context:
File "/tmp/.mount_buskilebOSKC/opt/python3.7/lib/python3.7/site-packages/usb1/__init__.py", line 2144, in __enter__
return self.open()
File "/tmp/.mount_buskilebOSKC/opt/python3.7/lib/python3.7/site-packages/usb1/__init__.py", line 2174, in open
loadLibrary()
File "/tmp/.mount_buskilebOSKC/opt/python3.7/lib/python3.7/site-packages/usb1/_libusb1.py", line 210, in loadLibrary
__loadLibrary(libusb)
File "/tmp/.mount_buskilebOSKC/opt/python3.7/lib/python3.7/site-packages/usb1/_libusb1.py", line 219, in __loadLibrary
libusb = __getLibrary()
File "/tmp/.mount_buskilebOSKC/opt/python3.7/lib/python3.7/site-packages/usb1/_libusb1.py", line 158, in __getLibrary
return dll_loader(filename, use_errno=True, use_last_error=True)
File "/tmp/.mount_buskilebOSKC/opt/python3.7/lib/python3.7/ctypes/__init__.py", line 364, in __init__
self._handle = _dlopen(self._name, mode)
OSError: libusb-1.0.so: cannot open shared object file: No such file or directory
root@75b34c9e3f49:~/buskill-app#
Actually, using the --privileged
argument, I was able to get this far in QubesOS as well (running a Debian container inside Debian inside QubesOS)
user@buskill:~$ export DOCKER_CONTENT_TRUST=1
user@buskill:~$ cd buskill-lin-v0.5.0-x86_64/
user@buskill:~/buskill-lin-v0.5.0-x86_64$ docker run --privileged --rm --cap-add "NET_ADMIN" -v "`pwd`:/root/buskill-app" -it --entrypoint /bin/bash debian:stable-slim
Unable to find image 'debian:stable-slim' locally
docker.io/library/debian@sha256:3d412a8a2e0f087738e6185c5cddab08827880a23e5bd9dc8b2494d89e4b33fd: Pulling from library/debian
2bc8f4ca4fd1: Pull complete
Digest: sha256:3d412a8a2e0f087738e6185c5cddab08827880a23e5bd9dc8b2494d89e4b33fd
Status: Downloaded newer image for debian@sha256:3d412a8a2e0f087738e6185c5cddab08827880a23e5bd9dc8b2494d89e4b33fd
Tagging debian@sha256:3d412a8a2e0f087738e6185c5cddab08827880a23e5bd9dc8b2494d89e4b33fd as debian:stable-slim
root@d47557d3c633:/#
root@d47557d3c633:/# apt-get update
Get:1 http://deb.debian.org/debian stable InRelease [116 kB]
Get:2 http://deb.debian.org/debian-security stable-security InRelease [48.4 kB]
Get:3 http://deb.debian.org/debian stable-updates InRelease [44.1 kB]
Get:4 http://deb.debian.org/debian stable/main amd64 Packages [8184 kB]
Get:5 http://deb.debian.org/debian-security stable-security/main amd64 Packages [186 kB]
Get:6 http://deb.debian.org/debian stable-updates/main amd64 Packages [6344 B]
Fetched 8585 kB in 14s (598 kB/s)
Reading package lists... Done
root@d47557d3c633:/# apt-get install libfuse2
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Suggested packages:
fuse
The following NEW packages will be installed:
libfuse2
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 130 kB of archives.
After this operation, 362 kB of additional disk space will be used.
Get:1 http://deb.debian.org/debian stable/main amd64 libfuse2 amd64 2.9.9-5 [130 kB]
Fetched 130 kB in 1s (137 kB/s)
debconf: delaying package configuration, since apt-utils is not installed
Selecting previously unselected package libfuse2:amd64.
(Reading database ... 6661 files and directories currently installed.)
Preparing to unpack .../libfuse2_2.9.9-5_amd64.deb ...
Unpacking libfuse2:amd64 (2.9.9-5) ...
Setting up libfuse2:amd64 (2.9.9-5) ...
Processing triggers for libc-bin (2.31-13+deb11u4) ...
root@d47557d3c633:/#
root@d47557d3c633:/# cd /root/buskill-app
root@d47557d3c633:~/buskill-app# ./buskill-v0.5.0.AppImage --arm
buskill version {'VERSION': 'v0.5.0', 'GITHUB_REF': 'refs/heads/v0.5.0', 'GITHUB_SHA': '996c0f6147eacc2999433fd1e187b198b1becdf6', 'SOURCE_DATE_EPOCH': '1658495593'}
DEBUG: EXE_PATH:|/root/buskill-app/buskill-v0.5.0.AppImage|
DEBUG: EXE_DIR:|/root/buskill-app|
DEBUG: EXE_FILE:|buskill-v0.5.0.AppImage|
DEBUG: APP_DIR:|/root/buskill-app|
DEBUG: APPS_DIR:|/root|
DEBUG: os.environ['PATH']:|/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/root/buskill-app:/root/buskill-app|
INFO: using DATA_DIR:|/root/.buskill|
DEBUG: attempting to arm BusKill via <bound method BusKill.armNix of <packages.buskill.BusKill object at 0x7bcac22e25d0>>()
INFO: BusKill is armed. Listening for removal event.
INFO: To disarm the CLI, exit with ^C or close this terminal
Process Process-1:
Traceback (most recent call last):
File "/tmp/.mount_buskilhcl5LI/opt/python3.7/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
self.run()
File "/tmp/.mount_buskilhcl5LI/opt/python3.7/lib/python3.7/multiprocessing/process.py", line 99, in run
self._target(*self._args, **self._kwargs)
File "/tmp/.mount_buskilhcl5LI/opt/src/packages/buskill/__init__.py", line 624, in armNix
with usb1.USBContext() as context:
File "/tmp/.mount_buskilhcl5LI/opt/python3.7/lib/python3.7/site-packages/usb1/__init__.py", line 2144, in __enter__
return self.open()
File "/tmp/.mount_buskilhcl5LI/opt/python3.7/lib/python3.7/site-packages/usb1/__init__.py", line 2174, in open
loadLibrary()
File "/tmp/.mount_buskilhcl5LI/opt/python3.7/lib/python3.7/site-packages/usb1/_libusb1.py", line 210, in loadLibrary
__loadLibrary(libusb)
File "/tmp/.mount_buskilhcl5LI/opt/python3.7/lib/python3.7/site-packages/usb1/_libusb1.py", line 219, in __loadLibrary
libusb = __getLibrary()
File "/tmp/.mount_buskilhcl5LI/opt/python3.7/lib/python3.7/site-packages/usb1/_libusb1.py", line 158, in __getLibrary
return dll_loader(filename, use_errno=True, use_last_error=True)
File "/tmp/.mount_buskilhcl5LI/opt/python3.7/lib/python3.7/ctypes/__init__.py", line 364, in __init__
self._handle = _dlopen(self._name, mode)
OSError: libusb-1.0.so: cannot open shared object file: No such file or directory
root@d47557d3c633:~/buskill-app#
I fixed this by installing the libusb-1.0-0-dev
package inside my docker container.
Damn, and that AppImage was supposed to be self-contained. I'm not sure if there's a fix here. Anyway, this debian-slim
was the first Linux "distro" I've encountered without this already installed, I guess.
root@d47557d3c633:~/buskill-app# apt-get -y install libusb-1.0-0-dev
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following additional packages will be installed:
libusb-1.0-0 libusb-1.0-doc
The following NEW packages will be installed:
libusb-1.0-0 libusb-1.0-0-dev libusb-1.0-doc
0 upgraded, 3 newly installed, 0 to remove and 0 not upgraded.
Need to get 332 kB of archives.
After this operation, 2090 kB of additional disk space will be used.
Get:1 http://deb.debian.org/debian stable/main amd64 libusb-1.0-0 amd64 2:1.0.24-3 [60.2 kB]
Get:2 http://deb.debian.org/debian stable/main amd64 libusb-1.0-0-dev amd64 2:1.0.24-3 [80.4 kB]
Get:3 http://deb.debian.org/debian stable/main amd64 libusb-1.0-doc all 2:1.0.24-3 [192 kB]
Fetched 332 kB in 1s (233 kB/s)
debconf: delaying package configuration, since apt-utils is not installed
Selecting previously unselected package libusb-1.0-0:amd64.
(Reading database ... 6674 files and directories currently installed.)
Preparing to unpack .../libusb-1.0-0_2%3a1.0.24-3_amd64.deb ...
Unpacking libusb-1.0-0:amd64 (2:1.0.24-3) ...
Selecting previously unselected package libusb-1.0-0-dev:amd64.
Preparing to unpack .../libusb-1.0-0-dev_2%3a1.0.24-3_amd64.deb ...
Unpacking libusb-1.0-0-dev:amd64 (2:1.0.24-3) ...
Selecting previously unselected package libusb-1.0-doc.
Preparing to unpack .../libusb-1.0-doc_2%3a1.0.24-3_all.deb ...
Unpacking libusb-1.0-doc (2:1.0.24-3) ...
Setting up libusb-1.0-doc (2:1.0.24-3) ...
Setting up libusb-1.0-0:amd64 (2:1.0.24-3) ...
Setting up libusb-1.0-0-dev:amd64 (2:1.0.24-3) ...
Processing triggers for libc-bin (2.31-13+deb11u4) ...
root@d47557d3c633:~/buskill-app# ./buskill-v0.5.0.AppImage --arm
buskill version {'VERSION': 'v0.5.0', 'GITHUB_REF': 'refs/heads/v0.5.0', 'GITHUB_SHA': '996c0f6147eacc2999433fd1e187b198b1becdf6', 'SOURCE_DATE_EPOCH': '1658495593'}
DEBUG: EXE_PATH:|/root/buskill-app/buskill-v0.5.0.AppImage|
DEBUG: EXE_DIR:|/root/buskill-app|
DEBUG: EXE_FILE:|buskill-v0.5.0.AppImage|
DEBUG: APP_DIR:|/root/buskill-app|
DEBUG: APPS_DIR:|/root|
DEBUG: os.environ['PATH']:|/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/root/buskill-app:/root/buskill-app|
INFO: using DATA_DIR:|/root/.buskill|
DEBUG: attempting to arm BusKill via <bound method BusKill.armNix of <packages.buskill.BusKill object at 0x787c609e6b90>>()
INFO: BusKill is armed. Listening for removal event.
INFO: To disarm the CLI, exit with ^C or close this terminal
Anyway, this is great progress! Now I can run the BusKill app (cli version) inside a linux docker container on my laptop.
Now I just need to figure out how to simulate a USB removal in that docker container and then I can automate a bunch of unit tests across tons of different Linux distros.
well, I found that I'm not able to trigger BusKill (or detect any hotplug events) on either the host AppVM in QubesOS nor its container. I tried using losetup
and by attaching a physical USB drive to the AppVm. I do see some output in dmesg -w
, but nothing in the BusKill App
Ah! I got it! I think the problem was that I was mounting just the block device of the USB drive (QubesOS best-practice to avoid malicious filesystem partitions) rather than the whole USB drive itself. When I tried attaching the webcam to my AppVM, this spat-out of my buskill app running inside the container inside the AppVM
root@buskill:~/buskill-app# ./buskill-v0.5.0.AppImage --arm
buskill version {'VERSION': 'v0.5.0', 'GITHUB_REF': 'refs/heads/v0.5.0', 'GITHUB_SHA': '996c0f6147eacc2999433fd1e187b198b1becdf6', 'SOURCE_DATE_EPOCH': '1658495593'}
DEBUG: EXE_PATH:|/root/buskill-app/buskill-v0.5.0.AppImage|
DEBUG: EXE_DIR:|/root/buskill-app|
DEBUG: EXE_FILE:|buskill-v0.5.0.AppImage|
DEBUG: APP_DIR:|/root/buskill-app|
DEBUG: APPS_DIR:|/root|
DEBUG: os.environ['PATH']:|/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/root/buskill-app:/root/buskill-app|
INFO: using DATA_DIR:|/root/.buskill|
DEBUG: attempting to arm BusKill via <bound method BusKill.armNix of <packages.buskill.BusKill object at 0x712761277b50>>()
INFO: BusKill is armed. Listening for removal event.
INFO: To disarm the CLI, exit with ^C or close this terminal
DEBUG: called hotplugCallbackNix()
context:|<usb1.USBContext object at 0x712760fb9e10>|
device:|Bus 001 Device 001: ID 1d6b:0002|
event:|1|
usb1.HOTPLUG_EVENT_DEVICE_LEFT:|2|
DEBUG: called hotplugCallbackNix()
context:|<usb1.USBContext object at 0x712760fb9e10>|
device:|Bus 002 Device 001: ID 1d6b:0003|
event:|1|
usb1.HOTPLUG_EVENT_DEVICE_LEFT:|2|
DEBUG: called hotplugCallbackNix()
context:|<usb1.USBContext object at 0x712760fb9e10>|
device:|Bus 001 Device 002: ID 5986:2115|
event:|1|
usb1.HOTPLUG_EVENT_DEVICE_LEFT:|2|
The above output just says that it detected something being plugged-in. The event is 1
, which does not match the constant usb1.HOTPLUG_EVENT_DEVICE_LEFT
, which is 2
.
I removed the webcam from the AppVM, and this is what I got
DEBUG: called hotplugCallbackNix()
context:|<usb1.USBContext object at 0x712760fb9e10>|
device:|Bus 001 Device 002: ID 5986:2115|
event:|2|
usb1.HOTPLUG_EVENT_DEVICE_LEFT:|2|
calling <bound method BusKill.triggerLin of <packages.buskill.BusKill object at 0x712761277b50>>
DEBUG: BusKill lockscreen trigger executing now
Success!
I tried to minimize the permissions needed for the docker container.
Unfortunately, I found that when I removed the --net host
argument, I could no longer catch usb hotplug removal events in the BusKill app that ran inside that docker container
# this won't work
docker run --privileged --rm --cap-add "NET_ADMIN" -v /run/udev/control:/run/udev/control -v "`pwd`:/root/buskill-app" -it --entrypoint /bin/bash debian:stable-slim
Unfortunately, I found that if I removed the --privileged
flag, then I'd get FUSE errors from running the AppImage again
# this won't work
docker run --rm --cap-add "NET_ADMIN" --net host -v /run/udev/control:/run/udev/control -v "`pwd`:/root/buskill-app" -it --entrypoint /bin/bash debian:stable-slim
Unfortunately, I found that when I removed the -v /run/udev/control:/run/udev/control
argument, I could no longer catch usb hotplug removal events in the BusKill app that ran inside that docker container
# this won't work
docker run --privileged --rm --cap-add "NET_ADMIN" --net host -v "`pwd`:/root/buskill-app" -it --entrypoint /bin/bash debian:stable-slim
I found that if I removed the --cap-add "NET_ADMIN"
flag, then I can still detect USB hotplug removal events inside the BusKill app that runs inside the docker container
# this does work
docker run --privileged --rm --net host -v /run/udev/control:/run/udev/control -v "`pwd`:/root/buskill-app" -it --entrypoint /bin/bash debian:stable-slim
iirc, the --cap-add "NET_ADMIN"
is something we use for the CI pipeline of BusKill builds. It allows the container to run iptables
so that we can prevent naughty, insecure package updater like brew
and pip
from trying to update our program's dependencies from the big, bad, dangerous internet (without safe integrity/authentication checks)
Therefore, we can run the following on a Linux system to test the BusKill app inside another dockerized linux system in a container
cd ${HOME}
wget https://github.com/BusKill/buskill-app/releases/download/v0.5.0/buskill-lin-v0.5.0-x86_64.tbz
tar -xjvf buskill-lin-v0.5.0-x86_64.tbz
sudo apt-get install docker.io
sudo bash -c 'gpasswd -a "${SUDO_USER}" docker'
su - `whoami`
cd ${HOME}/buskill-lin-v0.5.0-x86_64
export DOCKER_CONTENT_TRUST=1
docker run --privileged --rm --net host -v /run/udev/control:/run/udev/control -v "`pwd`:/root/buskill-app" -it --entrypoint /bin/bash debian:stable-slim
apt-get update
apt-get install -y libfuse2 libusb-1.0-0-dev
cd /root/buskill-app
./buskill-v0.5.0.AppImage --arm
...And then just insert a USB drive into the host system (additionally in QubesOS, attach the USB drive to the AppVM). Then remove it. BusKill should detect it.
There's no GUI in these containers, so we're not actually able to detect if the lockscreen is executing properly. But we can at least check that BusKill's usb hotplug detection is working.
And with that, I fired-up a new Debian 10 DispVM in Qubes, and I opened 8x terminals. In the first one, I typed
cd ${HOME}
wget https://github.com/BusKill/buskill-app/releases/download/v0.5.0/buskill-lin-v0.5.0-x86_64.tbz
tar -xjvf buskill-lin-v0.5.0-x86_64.tbz
sudo apt-get install docker.io
sudo bash -c 'gpasswd -a "${SUDO_USER}" docker'
Then in each of the 8x terminals, I typed one of the following blocks
# terminal 1, Ubuntu 22.04
su - `whoami`
cd ${HOME}/buskill-lin-v0.5.0-x86_64
export DOCKER_CONTENT_TRUST=1
docker run --privileged --rm --net host -v /run/udev/control:/run/udev/control -v "`pwd`:/root/buskill-app" -it --entrypoint /bin/bash ubuntu:22.04
# terminal 2, Ubuntu 21.10
su - `whoami`
cd ${HOME}/buskill-lin-v0.5.0-x86_64
export DOCKER_CONTENT_TRUST=1
docker run --privileged --rm --net host -v /run/udev/control:/run/udev/control -v "`pwd`:/root/buskill-app" -it --entrypoint /bin/bash ubuntu:21.10
# terminal 3, Ubuntu 21.04
su - `whoami`
cd ${HOME}/buskill-lin-v0.5.0-x86_64
export DOCKER_CONTENT_TRUST=1
docker run --privileged --rm --net host -v /run/udev/control:/run/udev/control -v "`pwd`:/root/buskill-app" -it --entrypoint /bin/bash ubuntu:21.04
# terminal 4, Ubuntu 20.10
su - `whoami`
cd ${HOME}/buskill-lin-v0.5.0-x86_64
export DOCKER_CONTENT_TRUST=1
docker run --privileged --rm --net host -v /run/udev/control:/run/udev/control -v "`pwd`:/root/buskill-app" -it --entrypoint /bin/bash ubuntu:20.10
# terminal 5, Ubuntu 20.04
su - `whoami`
cd ${HOME}/buskill-lin-v0.5.0-x86_64
export DOCKER_CONTENT_TRUST=1
docker run --privileged --rm --net host -v /run/udev/control:/run/udev/control -v "`pwd`:/root/buskill-app" -it --entrypoint /bin/bash ubuntu:20.04
# terminal 6, Ubuntu 18.04
su - `whoami`
cd ${HOME}/buskill-lin-v0.5.0-x86_64
export DOCKER_CONTENT_TRUST=1
docker run --privileged --rm --net host -v /run/udev/control:/run/udev/control -v "`pwd`:/root/buskill-app" -it --entrypoint /bin/bash ubuntu:18.04
# terminal 7, Ubuntu 16.04
su - `whoami`
cd ${HOME}/buskill-lin-v0.5.0-x86_64
export DOCKER_CONTENT_TRUST=1
docker run --privileged --rm --net host -v /run/udev/control:/run/udev/control -v "`pwd`:/root/buskill-app" -it --entrypoint /bin/bash ubuntu:16.04
# terminal 8, Ubuntu 14.04
su - `whoami`
cd ${HOME}/buskill-lin-v0.5.0-x86_64
export DOCKER_CONTENT_TRUST=1
docker run --privileged --rm --net host -v /run/udev/control:/run/udev/control -v "`pwd`:/root/buskill-app" -it --entrypoint /bin/bash ubuntu:14.04
Then in all of the terminals (each now with a bash session open for a different version of Ubuntu in a docker container), I ran the following to install the depends
apt-get update
apt-get install -y libfuse2 libusb-1.0-0-dev
Unfortunately, this didn't work for three of the Ubuntu versions. Curiously, just the ones in the middle complained that the packages couldn't be located. Actually, they all failed on the apt-get update
. Lots of 404 errors. Hmm. Here's the ones that couldn't get packages from apt:
On all the others, I was able to start BusKill with the following commands
cd /root/buskill-app
./buskill-v0.5.0.AppImage --arm
I attached my webcam to the DispVM host. All 5x of the Ubuntu containers detected the usb hotplug event.
I waited a few seconds, and then I detached the webcam. All 5x of the Ubuntu containers correctly detected the usb hotplug removal event, and said they were executing the lockscreen command. Fantasitc!
This process isn't exactly automated how I'd like it, but it's a hell of a lot faster than installing each distro on a machine and then testing it.
I'm closing this for now.
Found this cool project for linux systems:
I'm doing some testing for the v0.7.0 release. I spun up a Linux Mint HVM on my QubesOS dev laptop, but I couldn't figure out how to attach/remove a USB device from it (Qubes gave me Error: QubesVMError - Domain ... qrexec not connected
).
Note to self: I tried two ways to simulate USB removal events on a Linux system locked-up in VM; neither worked :(
udevadm test -a remove /sys/dev/block/$(ls /sys/dev/block | head -n1)
udevadm trigger -v -c remove /sys/dev/block/$(ls /sys/dev/block | head -n1)
dd if=/dev/zero of=zero bs=1M count=1
sudo losetup --show zero
sudo losetup -d /dev/loop0
This ticket will track the effort to add the project's first unit to our CI pipeline such that:
I'm actually not sure if [3] is possible with GitHub actions. Probably the first thing to determine is how the heck we can make a USB drive get removed from the OS on the GitHub runner.
I'll start with Linux. Once this is done, we should pile in as many popular Linux distros and versions as possible to see if BusKill fails with any of them.
After Linux, let's try Windows and MacOS.