Build Request

[Andrew-J-Larson]

Could you include builds for raspi 64 bit, in the releases?

[malmeloo]

GitHub actions doesn't support raspberry pi runners, but it might be possible using https://github.com/pguyot/arm-runner-action. I'll experiment a bit to see if that works, would you be willing to test it once I've got a binary built?

[Andrew-J-Larson]

Yeah sure I'm up for testing

[malmeloo]

It's taken some effort, but I finally got it to build...

https://github.com/RiiConnect24/RiiTag-RPC/suites/13286428326/artifacts/725078769

It's built in a pi 4b environment, but I don't think that matters as much as the architecture. Let me know if it works!

[Andrew-J-Larson]

It's taken some effort, but I finally got it to build...

https://github.com/RiiConnect24/RiiTag-RPC/suites/13286428326/artifacts/725078769

It's built in a pi 4b environment, but I don't think that matters as much as the architecture. Let me know if it works!

Tried it out, and bash doesn't seem to want to execute it, and sh (not bash) comes up with a not found error.

[Andrew-J-Larson]

Not sure if this helps, but I did notice that by default (at this time) the version of Python included in the repositories for the Raspberry Pi is 3.9.x something. If I remember correctly, while perusing through your code, I believe you're targeting 3.10.x something? In which case, you'll have to make sure that the Python version installed on the Raspberry Pi is the same version that you require for compiling.

[Andrew-J-Larson]

@DismissedGuy, here's a script I made (normally for updating), but modified here to just grab the repo at the latest release commit (assumes sudo apt update && sudo apt upgrade -y && sudo apt install python3 python3-pip -y has already been ran along with requirements.txt already installed), then compiles for a working executable (at least when tested on my physical Raspi 4b):

#!/bin/bash

cd "RiiTag-RPC-SOURCECODE"
if [ -z "$(ls -A)" ]; then
  git clone "https://github.com/RiiConnect24/RiiTag-RPC" .
else
  git pull
fi
git reset --hard 4b646fb # 5d96428
pip install -r requirements.txt
pyinstaller riitag-rpc.spec
cp -rf dist/riitag-rpc ../
cd ..
chmod +x riitag-rpc

And this is the output from it (notice the many "Cannot find" errors? not sure how much that affects the build, but I assume is related to using python 3.9.x instead of 3.10.x):

hint: Pulling without specifying how to reconcile divergent branches is
hint: discouraged. You can squelch this message by running one of the following
hint: commands sometime before your next pull:
hint: 
hint:   git config pull.rebase false  # merge (the default strategy)
hint:   git config pull.rebase true   # rebase
hint:   git config pull.ff only       # fast-forward only
hint: 
hint: You can replace "git config" with "git config --global" to set a default
hint: preference for all repositories. You can also pass --rebase, --no-rebase,
hint: or --ff-only on the command line to override the configured default per
hint: invocation.
Already up to date.
HEAD is now at 4b646fb Bump version
Defaulting to user installation because normal site-packages is not writeable
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Requirement already satisfied: requests in /home/drc-sim-user/.local/lib/python3.9/site-packages (from -r requirements.txt (line 1)) (2.28.1)
Requirement already satisfied: prompt_toolkit in /home/drc-sim-user/.local/lib/python3.9/site-packages (from -r requirements.txt (line 2)) (3.0.36)
Requirement already satisfied: pypresence in /home/drc-sim-user/.local/lib/python3.9/site-packages (from -r requirements.txt (line 3)) (4.2.1)
Requirement already satisfied: nest_asyncio in /home/drc-sim-user/.local/lib/python3.9/site-packages (from -r requirements.txt (line 4)) (1.5.6)
Requirement already satisfied: sentry_sdk==0.16.1 in /home/drc-sim-user/.local/lib/python3.9/site-packages (from -r requirements.txt (line 5)) (0.16.1)
Requirement already satisfied: certifi in /home/drc-sim-user/.local/lib/python3.9/site-packages (from sentry_sdk==0.16.1->-r requirements.txt (line 5)) (2022.12.7)
Requirement already satisfied: urllib3>=1.10.0 in /home/drc-sim-user/.local/lib/python3.9/site-packages (from sentry_sdk==0.16.1->-r requirements.txt (line 5)) (1.26.13)
Requirement already satisfied: charset-normalizer<3,>=2 in /home/drc-sim-user/.local/lib/python3.9/site-packages (from requests->-r requirements.txt (line 1)) (2.1.1)
Requirement already satisfied: idna<4,>=2.5 in /home/drc-sim-user/.local/lib/python3.9/site-packages (from requests->-r requirements.txt (line 1)) (3.4)
Requirement already satisfied: wcwidth in /home/drc-sim-user/.local/lib/python3.9/site-packages (from prompt_toolkit->-r requirements.txt (line 2)) (0.2.5)
530 INFO: PyInstaller: 5.7.0
530 INFO: Python: 3.9.2
539 INFO: Platform: Linux-6.1.21-v8+-aarch64-with-glibc2.31
548 INFO: UPX is not available.
552 INFO: Extending PYTHONPATH with paths
['/home/drc-sim-user/Documents/RiiTag-RPC/RiiTag-RPC-SOURCECODE']
pygame 2.1.2 (SDL 2.0.16, Python 3.9.2)
Hello from the pygame community. https://www.pygame.org/contribute.html
1882 INFO: checking Analysis
1883 INFO: Building Analysis because Analysis-00.toc is non existent
1883 INFO: Initializing module dependency graph...
1887 INFO: Caching module graph hooks...
1908 WARNING: Several hooks defined for module 'numpy'. Please take care they do not conflict.
1916 INFO: Analyzing base_library.zip ...
6170 INFO: Loading module hook 'hook-encodings.py' from '/home/drc-sim-user/.local/lib/python3.9/site-packages/PyInstaller/hooks'...
10012 INFO: Loading module hook 'hook-pickle.py' from '/home/drc-sim-user/.local/lib/python3.9/site-packages/PyInstaller/hooks'...
12478 INFO: Loading module hook 'hook-heapq.py' from '/home/drc-sim-user/.local/lib/python3.9/site-packages/PyInstaller/hooks'...
14410 INFO: Caching module dependency graph...
14800 INFO: running Analysis Analysis-00.toc
14881 INFO: Analyzing /home/drc-sim-user/Documents/RiiTag-RPC/RiiTag-RPC-SOURCECODE/start.py
15199 INFO: Loading module hook 'hook-platform.py' from '/home/drc-sim-user/.local/lib/python3.9/site-packages/PyInstaller/hooks'...
16043 INFO: Loading module hook 'hook-multiprocessing.util.py' from '/home/drc-sim-user/.local/lib/python3.9/site-packages/PyInstaller/hooks'...
16566 INFO: Loading module hook 'hook-xml.py' from '/home/drc-sim-user/.local/lib/python3.9/site-packages/PyInstaller/hooks'...
19005 INFO: Loading module hook 'hook-sentry_sdk.py' from '/home/drc-sim-user/.local/lib/python3.9/site-packages/_pyinstaller_hooks_contrib/hooks/stdhooks'...
20251 INFO: Processing pre-safe import module hook urllib3.packages.six.moves from '/home/drc-sim-user/.local/lib/python3.9/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-urllib3.packages.six.moves.py'.
22478 INFO: Loading module hook 'hook-certifi.py' from '/home/drc-sim-user/.local/lib/python3.9/site-packages/_pyinstaller_hooks_contrib/hooks/stdhooks'...
24592 INFO: Loading module hook 'hook-wcwidth.py' from '/home/drc-sim-user/.local/lib/python3.9/site-packages/PyInstaller/hooks'...
25216 INFO: Loading module hook 'hook-pygments.py' from '/home/drc-sim-user/.local/lib/python3.9/site-packages/PyInstaller/hooks'...
28602 INFO: Loading module hook 'hook-importlib_metadata.py' from '/home/drc-sim-user/.local/lib/python3.9/site-packages/PyInstaller/hooks'...
29189 INFO: Loading module hook 'hook-pkg_resources.py' from '/home/drc-sim-user/.local/lib/python3.9/site-packages/PyInstaller/hooks'...
31362 INFO: Loading module hook 'hook-sysconfig.py' from '/home/drc-sim-user/.local/lib/python3.9/site-packages/PyInstaller/hooks'...
33931 INFO: Loading module hook 'hook-xml.dom.domreg.py' from '/home/drc-sim-user/.local/lib/python3.9/site-packages/PyInstaller/hooks'...
39938 INFO: Loading module hook 'hook-cryptography.py' from '/home/drc-sim-user/.local/lib/python3.9/site-packages/_pyinstaller_hooks_contrib/hooks/stdhooks'...
46188 INFO: Processing module hooks...
49245 INFO: Loading module hook 'hook-jinja2.py' from '/home/drc-sim-user/.local/lib/python3.9/site-packages/_pyinstaller_hooks_contrib/hooks/stdhooks'...
51456 INFO: Loading module hook 'hook-difflib.py' from '/home/drc-sim-user/.local/lib/python3.9/site-packages/PyInstaller/hooks'...
53398 INFO: Loading module hook 'hook-PIL.py' from '/home/drc-sim-user/.local/lib/python3.9/site-packages/PyInstaller/hooks'...
53638 INFO: Loading module hook 'hook-PIL.Image.py' from '/home/drc-sim-user/.local/lib/python3.9/site-packages/PyInstaller/hooks'...
55134 INFO: Loading module hook 'hook-pycparser.py' from '/home/drc-sim-user/.local/lib/python3.9/site-packages/_pyinstaller_hooks_contrib/hooks/stdhooks'...
56652 INFO: Processing pre-find module path hook distutils from '/home/drc-sim-user/.local/lib/python3.9/site-packages/PyInstaller/hooks/pre_find_module_path/hook-distutils.py'.
58066 INFO: Loading module hook 'hook-distutils.py' from '/home/drc-sim-user/.local/lib/python3.9/site-packages/PyInstaller/hooks'...
58233 INFO: Loading module hook 'hook-distutils.util.py' from '/home/drc-sim-user/.local/lib/python3.9/site-packages/PyInstaller/hooks'...
58908 INFO: Loading module hook 'hook-setuptools.py' from '/home/drc-sim-user/.local/lib/python3.9/site-packages/PyInstaller/hooks'...
67486 INFO: Loading module hook 'hook-numpy.py' from '/home/drc-sim-user/.local/lib/python3.9/site-packages/numpy/_pyinstaller'...
69008 INFO: Loading module hook 'hook-numpy._pytesttester.py' from '/home/drc-sim-user/.local/lib/python3.9/site-packages/PyInstaller/hooks'...
72053 INFO: Loading module hook 'hook-xml.etree.cElementTree.py' from '/home/drc-sim-user/.local/lib/python3.9/site-packages/PyInstaller/hooks'...
75413 INFO: Loading module hook 'hook-PIL.ImageFilter.py' from '/home/drc-sim-user/.local/lib/python3.9/site-packages/PyInstaller/hooks'...
82439 INFO: Loading module hook 'hook-PIL.SpiderImagePlugin.py' from '/home/drc-sim-user/.local/lib/python3.9/site-packages/PyInstaller/hooks'...
82695 INFO: Processing pre-safe import module hook win32com from '/home/drc-sim-user/.local/lib/python3.9/site-packages/_pyinstaller_hooks_contrib/hooks/pre_safe_import_module/hook-win32com.py'.
85107 INFO: Loading module hook 'hook-setuptools.msvc.py' from '/home/drc-sim-user/.local/lib/python3.9/site-packages/PyInstaller/hooks'...
88727 INFO: Loading module hook 'hook-docutils.py' from '/home/drc-sim-user/.local/lib/python3.9/site-packages/_pyinstaller_hooks_contrib/hooks/stdhooks'...
99242 INFO: Looking for ctypes DLLs
99582 INFO: Analyzing run-time hooks ...
99642 INFO: Including run-time hook '/home/drc-sim-user/.local/lib/python3.9/site-packages/PyInstaller/hooks/rthooks/pyi_rth_pkgutil.py'
99650 INFO: Including run-time hook '/home/drc-sim-user/.local/lib/python3.9/site-packages/PyInstaller/hooks/rthooks/pyi_rth_inspect.py'
99654 INFO: Including run-time hook '/home/drc-sim-user/.local/lib/python3.9/site-packages/PyInstaller/hooks/rthooks/pyi_rth_multiprocessing.py'
99663 INFO: Including run-time hook '/home/drc-sim-user/.local/lib/python3.9/site-packages/PyInstaller/hooks/rthooks/pyi_rth_pkgres.py'
99678 INFO: Including run-time hook '/home/drc-sim-user/.local/lib/python3.9/site-packages/PyInstaller/hooks/rthooks/pyi_rth_setuptools.py'
99789 INFO: Looking for dynamic libraries
/home/drc-sim-user/.local/lib/python3.9/site-packages/PyInstaller/building/build_main.py:174: UserWarning: The numpy.array_api submodule is still experimental. See NEP 47.
  __import__(package)
4108 WARNING: Cannot find libpng16-19589d9b.so.16.38.0 (needed by /home/drc-sim-user/.local/lib/python3.9/site-packages/PIL/../Pillow.libs/libfreetype-7332d18b.so.6.18.3)
4138 WARNING: Cannot find libfreetype-7332d18b.so.6.18.3 (needed by /home/drc-sim-user/.local/lib/python3.9/site-packages/PIL/../Pillow.libs/libharfbuzz-e3a398a7.so.0.50301.0)
4198 WARNING: Cannot find libwebp-e036d857.so.7.1.5 (needed by /home/drc-sim-user/.local/lib/python3.9/site-packages/PIL/../Pillow.libs/libwebpmux-ad749361.so.3.0.10)
4227 WARNING: Cannot find libwebp-e036d857.so.7.1.5 (needed by /home/drc-sim-user/.local/lib/python3.9/site-packages/PIL/../Pillow.libs/libwebpdemux-a8adc337.so.2.0.11)
4376 WARNING: Cannot find libgfortran-daac5196.so.5.0.0 (needed by /home/drc-sim-user/.local/lib/python3.9/site-packages/numpy/core/../../numpy.libs/libopenblas64_p-r0-cecebdce.3.21.so)
4466 WARNING: Cannot find libXau-7926f62a.so.6.0.0 (needed by /home/drc-sim-user/.local/lib/python3.9/site-packages/PIL/../Pillow.libs/libxcb-e417d7b4.so.1.1.0)
4496 WARNING: Cannot find liblzma-d172858b.so.5.2.7 (needed by /home/drc-sim-user/.local/lib/python3.9/site-packages/PIL/../Pillow.libs/libtiff-4d622be8.so.5.8.0)
4497 WARNING: Cannot find libjpeg-33660c83.so.62.3.0 (needed by /home/drc-sim-user/.local/lib/python3.9/site-packages/PIL/../Pillow.libs/libtiff-4d622be8.so.5.8.0)
105239 INFO: Looking for eggs
105240 INFO: Python library not in binary dependencies. Doing additional searching...
105301 INFO: Using Python library /lib/aarch64-linux-gnu/libpython3.9.so.1.0
105372 INFO: Warnings written to /home/drc-sim-user/Documents/RiiTag-RPC/RiiTag-RPC-SOURCECODE/build/riitag-rpc/warn-riitag-rpc.txt
105829 INFO: Graph cross-reference written to /home/drc-sim-user/Documents/RiiTag-RPC/RiiTag-RPC-SOURCECODE/build/riitag-rpc/xref-riitag-rpc.html
105983 INFO: Appending 'datas' from .spec
106034 INFO: checking PYZ
106034 INFO: Building PYZ because PYZ-00.toc is non existent
106034 INFO: Building PYZ (ZlibArchive) /home/drc-sim-user/Documents/RiiTag-RPC/RiiTag-RPC-SOURCECODE/build/riitag-rpc/PYZ-00.pyz
110985 INFO: Building PYZ (ZlibArchive) /home/drc-sim-user/Documents/RiiTag-RPC/RiiTag-RPC-SOURCECODE/build/riitag-rpc/PYZ-00.pyz completed successfully.
111111 INFO: checking PKG
111111 INFO: Building PKG because PKG-00.toc is non existent
111112 INFO: Building PKG (CArchive) riitag-rpc.pkg
137034 INFO: Building PKG (CArchive) riitag-rpc.pkg completed successfully.
137053 INFO: Bootloader /home/drc-sim-user/.local/lib/python3.9/site-packages/PyInstaller/bootloader/Linux-64bit-arm/run
137053 INFO: checking EXE
137054 INFO: Building EXE because EXE-00.toc is non existent
137054 INFO: Building EXE from EXE-00.toc
137054 INFO: Copying bootloader EXE to /home/drc-sim-user/Documents/RiiTag-RPC/RiiTag-RPC-SOURCECODE/dist/riitag-rpc
137055 INFO: Appending PKG archive to custom ELF section in EXE
137299 INFO: Building EXE from EXE-00.toc completed successfully.

[Andrew-J-Larson]

By the way, when I run my script the compiled file is executable and so far with no noticable errors, but I've yet to extensively test that 1.3.1 version.

[malmeloo]

It's taken some effort, but I finally got it to build... RiiConnect24/RiiTag-RPC/suites/13286428326/artifacts/725078769 It's built in a pi 4b environment, but I don't think that matters as much as the architecture. Let me know if it works!

Tried it out, and bash doesn't seem to want to execute it, and sh (not bash) comes up with a not found error.

You shouldn't run it with bash or sh, as it's not a shell script. You should be able to just run it as a regular executable (./riitag-rpc).

Not sure if this helps, but I did notice that by default (at this time) the version of Python included in the repositories for the Raspberry Pi is 3.9.x something. If I remember correctly, while perusing through your code, I believe you're targeting 3.10.x something? In which case, you'll have to make sure that the Python version installed on the Raspberry Pi is the same version that you require for compiling.

I noticed that as well, but AFAIK there is nothing in the codebase that is explicitly only supported on python 3.10+. I'm using py3.10 for the actions workflow simply because it's newer and should be backwards compatible (although I believe 3.11 support is also already available for runners).

And this is the output from it (notice the many "Cannot find" errors? not sure how much that affects the build, but I assume is related to using python 3.9.x instead of 3.10.x):

I'm not getting those errors when building via github actions: https://github.com/RiiConnect24/RiiTag-RPC/actions/runs/5138796135/jobs/9248501336 The messages seem to suggest that those libs are requested by PIL and numpy, which I don't think are depended on by this project. So this is probably an issue with your environment that shouldn't affect the program itself.

[Andrew-J-Larson]

It's taken some effort, but I finally got it to build... RiiConnect24/RiiTag-RPC/suites/13286428326/artifacts/725078769 It's built in a pi 4b environment, but I don't think that matters as much as the architecture. Let me know if it works!

Tried it out, and bash doesn't seem to want to execute it, and sh (not bash) comes up with a not found error.

You shouldn't run it with bash or sh, as it's not a shell script. You should be able to just run it as a regular executable (./riitag-rpc).

Not sure if this helps, but I did notice that by default (at this time) the version of Python included in the repositories for the Raspberry Pi is 3.9.x something. If I remember correctly, while perusing through your code, I believe you're targeting 3.10.x something? In which case, you'll have to make sure that the Python version installed on the Raspberry Pi is the same version that you require for compiling.

I noticed that as well, but AFAIK there is nothing in the codebase that is explicitly only supported on python 3.10+. I'm using py3.10 for the actions workflow simply because it's newer and should be backwards compatible (although I believe 3.11 support is also already available for runners).

And this is the output from it (notice the many "Cannot find" errors? not sure how much that affects the build, but I assume is related to using python 3.9.x instead of 3.10.x):

I'm not getting those errors when building via github actions: https://github.com/RiiConnect24/RiiTag-RPC/actions/runs/5138796135/jobs/9248501336 The messages seem to suggest that those libs are requested by PIL and numpy, which I don't think are depended on by this project. So this is probably an issue with your environment that shouldn't affect the program itself.

Everything else you said I noted, but yes, that is exactly how I'm running the executable (./riitag-rpc executes out of sh). Execution through bash shouldn't be different, but seeing how there was a different error, thought I should note it (ran with bash ./riitag-rpc).

[Andrew-J-Larson]

Note, with my compiled executable, running the file either way (./riitag-rpc or bash ./riitag-rpc) works just fine.

[malmeloo]

That's interesting, because when I compile it on my ubuntu system I'm getting a regular executable:

❯ file riitag-rpc-ubuntu
riitag-rpc-ubuntu: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=423f791d8a7fd6c80919698fa0f9e214248472f5, for GNU/Linux 2.6.32, stripped

❯ file riitag-rpc-rpi
riitag-rpc-rpi: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-armhf.so.3, for GNU/Linux 3.2.0, BuildID[sha1]=de9fc9e0d4e651a4fab014dc13f0fa4d3846b9d9, stripped

❯ bash riitag-rpc-ubuntu
riitag-rpc-ubuntu: riitag-rpc-ubuntu: cannot execute binary file

❯ sh riitag-rpc-ubuntu
riitag-rpc-ubuntu: 1: Syntax error: ")" unexpected

Could you upload the file you built on your pi?

I did just notice that it seems to have built a 32-bit executable, but that shouldn't matter since 64-bit rpi os can run 32-bit binaries just fine to my knowledge. What os are you running?

[malmeloo]

I just built a 64 bit executable, maybe this one works?

https://github.com/RiiConnect24/RiiTag-RPC/suites/13326983162/artifacts/728253589

[Andrew-J-Larson]

That's interesting, because when I compile it on my ubuntu system I'm getting a regular executable:

❯ file riitag-rpc-ubuntu
riitag-rpc-ubuntu: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=423f791d8a7fd6c80919698fa0f9e214248472f5, for GNU/Linux 2.6.32, stripped

❯ file riitag-rpc-rpi
riitag-rpc-rpi: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-armhf.so.3, for GNU/Linux 3.2.0, BuildID[sha1]=de9fc9e0d4e651a4fab014dc13f0fa4d3846b9d9, stripped

❯ bash riitag-rpc-ubuntu
riitag-rpc-ubuntu: riitag-rpc-ubuntu: cannot execute binary file

❯ sh riitag-rpc-ubuntu
riitag-rpc-ubuntu: 1: Syntax error: ")" unexpected

Could you upload the file you built on your pi?

I did just notice that it seems to have built a 32-bit executable, but that shouldn't matter since 64-bit rpi os can run 32-bit binaries just fine to my knowledge. What os are you running?

I'm running Raspberry Pi OS 64-bit. And yeah I'll go ahead and upload it here in a bit, if after testing your latest executable file doesn't work.

[Andrew-J-Larson]

I have a hunch that Raspberry Pi OS (64-bit) doesn't come with the 32 bit libraries needed to properly generate 32 binaries (which is mostly likely why your 32 bit version wasn't working). Or, 32 bit support isn't enabled from base install, which means it would need to be enabled first for that 32 bit version to work. Or, it could even be a combination of both.

I just built a 64 bit executable, maybe this one works?

https://github.com/RiiConnect24/RiiTag-RPC/suites/13326983162/artifacts/728253589

Anyways though, this one worked, and the SHA1 hash matches my compiled executable file. 👌🏻

• SHA1: bca5e3d07e62491b35ec338fcd775d7ebc17540c

[malmeloo]

It was actually generating 32 bit binaries because I was accidentally building it on the 32-bit version of rpi os... so no idea why it wouldn't run on your pi, but hey, nice to see that it works!

I'd still like to make some other workflow changes, so I'll probably merge the related pr after that and then attach the new binaries to the latest release.