Closed KenwoodFox closed 1 year ago
This might not be needed since zope.interfaces
is already there :o
Looks like crypto may be needed too, mustive forgot that
❯ make deploy
python robot.py deploy --nc
10:41:48:977 INFO : wpilib : WPILib version 2022.4.1.6
10:41:48:977 INFO : wpilib : HAL version 2022.4.1.2
10:41:48:977 INFO : wpilib : Running with simulated HAL.
10:41:48:978 INFO : wpilib : robotpy-wpimath version 2022.4.1.3
10:41:48:978 INFO : wpilib : robotpy-wpiutil version 2022.4.1.2
10:41:48:978 INFO : wpilib : robotpy-halsim-gui version 2022.4.1.1
10:41:48:978 INFO : wpilib : robotpy-commands-v2 version 2022.4.1.1
10:41:48:978 INFO : wpilib : pyntcore version 2022.4.1.0
10:41:48:978 INFO : wpilib : robotpy-ctre version 2022.1.1
10:41:48:978 INFO : wpilib : robotpy-rev version 2022.0.1
10:41:48:978 INFO : faulthandler : registered SIGUSR2 for PID 880250
10:41:48:978 INFO : robotpy.installer : Finding robot for team 1721
10:41:48:980 INFO : robotpy.installer : -> Robot is at 10.17.21.2
10:41:48:980 INFO : robotpy.installer : Connecting to robot via SSH at 10.17.21.2
10:41:49:178 INFO : paramiko.transport : Connected (version 2.0, client OpenSSH_7.6)
10:41:49:301 INFO : paramiko.transport : Auth banner: b'NI Linux Real-Time (run mode)\n\nLog in with your NI-Auth credentials.\n\n'
10:41:49:301 INFO : paramiko.transport : Authentication (password) successful!
RoboRIO has WPILib version 2022.4.1.6
10:41:50:239 INFO : paramiko.transport.sftp: [chan 6] Opened sftp connection (server version 3)
make /home/lvuser/py_new
deploy.json -> /home/lvuser/py_new/deploy.json
robot_requirements.txt -> /home/lvuser/py_new/robot_requirements.txt
README.md -> /home/lvuser/py_new/README.md
...
constants/robot_pid.yaml -> /home/lvuser/py_new/constants/robot_pid.yaml
constants/constants.py -> /home/lvuser/py_new/constants/constants.py
constants/robot_hardware.yaml -> /home/lvuser/py_new/constants/robot_hardware.yaml
constants/__init__.py -> /home/lvuser/py_new/constants/__init__.py
constants/robot_controls.yaml -> /home/lvuser/py_new/constants/robot_controls.yaml
10:41:50:467 INFO : paramiko.transport.sftp: [chan 6] sftp session closed.
Connecting to 10.17.21.2:1741...OK
10:41:50:470 INFO : deploy : Netconsole is listening...
[0.00] Traceback (most recent call last):
[0.00] File "/home/lvuser/py/robot.py", line 6, in <module>
[0.00] import roslibpy
[0.01] File "/usr/local/lib/python3.10/site-packages/roslibpy/__init__.py", line 113, in <module>
[0.01] from .ros import (
[0.01] File "/usr/local/lib/python3.10/site-packages/roslibpy/ros.py", line 11, in <module>
[0.01] from .comm import RosBridgeClientFactory
[0.01] File "/usr/local/lib/python3.10/site-packages/roslibpy/comm/__init__.py", line 9, in <module>
[0.01] from .comm_autobahn import AutobahnRosBridgeClientFactory as RosBridgeClientFactory
[0.01] File "/usr/local/lib/python3.10/site-packages/roslibpy/comm/comm_autobahn.py", line 6, in <module>
[0.01] from autobahn.twisted.websocket import WebSocketClientFactory
[0.01] File "/usr/local/lib/python3.10/site-packages/autobahn/twisted/__init__.py", line 58, in <module>
[0.01] from autobahn.twisted.wamp import ApplicationSession
[0.01] File "/usr/local/lib/python3.10/site-packages/autobahn/twisted/wamp.py", line 50, in <module>
[0.02] from autobahn.wamp import protocol, auth
[0.02] File "/usr/local/lib/python3.10/site-packages/autobahn/wamp/auth.py", line 43, in <module>
[0.02] from cryptography.hazmat.primitives.kdf.pbkdf2 import PBKDF2HMAC
[0.02] ModuleNotFoundError: No module named 'cryptography'
^CTraceback (most recent call last):
File "/home/joe/git/1721-RapidReact/rio/robot.py", line 143, in <module>
wpilib.run(BurntToaster)
File "/home/joe/.local/share/virtualenvs/rio-XHms6B9Y/lib/python3.10/site-packages/wpilib/_impl/main.py", line 210, in run
retval = options.cmdobj.run(options, robot_class, **kwargs)
File "/home/joe/.local/share/virtualenvs/rio-XHms6B9Y/lib/python3.10/site-packages/pyfrc/mains/cli_deploy.py", line 202, in run
if not self._do_deploy(ssh, options, robot_filename, robot_path):
File "/home/joe/.local/share/virtualenvs/rio-XHms6B9Y/lib/python3.10/site-packages/pyfrc/mains/cli_deploy.py", line 493, in _do_deploy
nc_thread.join()
File "/usr/lib/python3.10/threading.py", line 1089, in join
self._wait_for_tstate_lock()
File "/usr/lib/python3.10/threading.py", line 1109, in _wait_for_tstate_lock
if lock.acquire(block, timeout):
KeyboardInterrupt
make: *** [Makefile:27: deploy] Interrupt```
Hope its ok to take notes here :3
_init_posix
looks like python2
I was able to build this locally using python -m pip --disable-pip-version-check -v wheel --no-binary :all: -w dist roslibpy==1.3.0
but i did need a rust compiler, interesting.
Main issue still seems to be with setuputils, im sure this is a python 2.7 -> 3.x issue but i cant seem to find where this is introduced and by what package, twisted
?
Downloading setuptools-61.3.1.tar.gz (2.5 MB)
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'error'
ERROR: Command errored out with exit status 1:
command: /build/venv/cross/bin/python /tmp/tmplbk0p0p3_in_process.py get_requires_for_build_wheel /tmp/tmpdkkm7b7_
cwd: /tmp/pip-install-yp70ngq6/setuptools_56b28e538f6a40feae20110114aecc15
Complete output (28 lines):
Traceback (most recent call last):
File "/tmp/tmplbk0p0p3_in_process.py", line 349, in <module>
main()
File "/tmp/tmplbk0p0p3_in_process.py", line 331, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "/tmp/tmplbk0p0p3_in_process.py", line 117, in get_requires_for_build_wheel
return hook(config_settings)
File "/tmp/pip-install-yp70ngq6/setuptools_56b28e538f6a40feae20110114aecc15/setuptools/build_meta.py", line 177, in get_requires_for_build_wheel
return self._get_build_requires(
File "/tmp/pip-install-yp70ngq6/setuptools_56b28e538f6a40feae20110114aecc15/setuptools/build_meta.py", line 159, in _get_build_requires
self.run_setup()
File "/tmp/pip-install-yp70ngq6/setuptools_56b28e538f6a40feae20110114aecc15/setuptools/build_meta.py", line 174, in run_setup
exec(compile(code, __file__, 'exec'), locals())
File "setup.py", line 8, in <module>
from setuptools.command.install import install
File "/tmp/pip-install-yp70ngq6/setuptools_56b28e538f6a40feae20110114aecc15/setuptools/command/install.py", line 6, in <module>
import distutils.command.install as orig
File "/tmp/pip-install-yp70ngq6/setuptools_56b28e538f6a40feae20110114aecc15/setuptools/_distutils/command/install.py", line 14, in <module>
from distutils.sysconfig import get_config_vars
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
File "/build/venv/lib/site.py", line 74, in exec_module
_patch_module(module, self.patch)
File "/build/venv/lib/site.py", line 55, in _patch_module
exec(src, module.__dict__, module.__dict__)
File "<string>", line 13, in <module>
NameError: name '_init_posix' is not defined```
Just to reduce the spam Ive been trying to run these locally too but i get different errors running the docker container here than on github, working to sync the two.
Piwheels has arm builds, i tried to bypass crypto compiling by using pip install roslibpy --extra-index-url=https://www.piwheels.org/simple
but it downloaded the .whl
for every other package except crypto! https://www.piwheels.org/project/cryptography/
Traced it back to missing ffi.h
in cffi
(cross) root@baefa56118ca:/# pip install cffi
Looking in links: https://www.tortall.net/~robotpy/wheels/2022/roborio
Collecting cffi
Using cached cffi-1.15.0.tar.gz (484 kB)
Requirement already satisfied: pycparser in /build/venv/cross/lib/python3.10/site-packages (from cffi) (2.21)
Building wheels for collected packages: cffi
Building wheel for cffi (setup.py) ... error
ERROR: Command errored out with exit status 1:
command: /build/venv/cross/bin/python3.10 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-8fd4t_d1/cffi_240eb1f0a97a43c29e5c874656e69ce2/setup.py'"'"'; __file__='"'"'/tmp/pip-install-8fd4t_d1/cffi_240eb1f0a97a43c29e5c874656e69ce2/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-vyrzxrac
cwd: /tmp/pip-install-8fd4t_d1/cffi_240eb1f0a97a43c29e5c874656e69ce2/
Complete output (36 lines):
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-armv7l-3.10
creating build/lib.linux-armv7l-3.10/cffi
copying cffi/__init__.py -> build/lib.linux-armv7l-3.10/cffi
copying cffi/commontypes.py -> build/lib.linux-armv7l-3.10/cffi
copying cffi/vengine_gen.py -> build/lib.linux-armv7l-3.10/cffi
copying cffi/cffi_opcode.py -> build/lib.linux-armv7l-3.10/cffi
copying cffi/lock.py -> build/lib.linux-armv7l-3.10/cffi
copying cffi/backend_ctypes.py -> build/lib.linux-armv7l-3.10/cffi
copying cffi/setuptools_ext.py -> build/lib.linux-armv7l-3.10/cffi
copying cffi/error.py -> build/lib.linux-armv7l-3.10/cffi
copying cffi/verifier.py -> build/lib.linux-armv7l-3.10/cffi
copying cffi/recompiler.py -> build/lib.linux-armv7l-3.10/cffi
copying cffi/api.py -> build/lib.linux-armv7l-3.10/cffi
copying cffi/vengine_cpy.py -> build/lib.linux-armv7l-3.10/cffi
copying cffi/cparser.py -> build/lib.linux-armv7l-3.10/cffi
copying cffi/ffiplatform.py -> build/lib.linux-armv7l-3.10/cffi
copying cffi/pkgconfig.py -> build/lib.linux-armv7l-3.10/cffi
copying cffi/model.py -> build/lib.linux-armv7l-3.10/cffi
copying cffi/_cffi_include.h -> build/lib.linux-armv7l-3.10/cffi
copying cffi/parse_c_type.h -> build/lib.linux-armv7l-3.10/cffi
copying cffi/_embedding.h -> build/lib.linux-armv7l-3.10/cffi
copying cffi/_cffi_errors.h -> build/lib.linux-armv7l-3.10/cffi
running build_ext
building '_cffi_backend' extension
creating build/temp.linux-armv7l-3.10
creating build/temp.linux-armv7l-3.10/c
arm-frc2022-linux-gnueabi-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -DUSE__THREAD -DHAVE_SYNC_SYNCHRONIZE -I/usr/include/ffi -I/usr/include/libffi -I/build/venv/cross/include -I/build/crosspy/include/python3.10 -c c/_cffi_backend.c -o build/temp.linux-armv7l-3.10/c/_cffi_backend.o
c/_cffi_backend.c:15:10: fatal error: ffi.h: No such file or directory
#include <ffi.h>
^~~~~~~
compilation terminated.
error: command '/usr/local/bin/arm-frc2022-linux-gnueabi-gcc' failed with exit code 1
----------------------------------------
ERROR: Failed building wheel for cffi
Running setup.py clean for cffi
Failed to build cffi
Installing collected packages: cffi
Running setup.py install for cffi ... error
ERROR: Command errored out with exit status 1:
command: /build/venv/cross/bin/python3.10 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-8fd4t_d1/cffi_240eb1f0a97a43c29e5c874656e69ce2/setup.py'"'"'; __file__='"'"'/tmp/pip-install-8fd4t_d1/cffi_240eb1f0a97a43c29e5c874656e69ce2/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-j3ttetfm/install-record.txt --single-version-externally-managed --compile --install-headers /build/venv/cross/include/site/python3.10/cffi
cwd: /tmp/pip-install-8fd4t_d1/cffi_240eb1f0a97a43c29e5c874656e69ce2/
Complete output (36 lines):
running install
running build
running build_py
creating build
creating build/lib.linux-armv7l-3.10
creating build/lib.linux-armv7l-3.10/cffi
copying cffi/__init__.py -> build/lib.linux-armv7l-3.10/cffi
copying cffi/commontypes.py -> build/lib.linux-armv7l-3.10/cffi
copying cffi/vengine_gen.py -> build/lib.linux-armv7l-3.10/cffi
copying cffi/cffi_opcode.py -> build/lib.linux-armv7l-3.10/cffi
copying cffi/lock.py -> build/lib.linux-armv7l-3.10/cffi
copying cffi/backend_ctypes.py -> build/lib.linux-armv7l-3.10/cffi
copying cffi/setuptools_ext.py -> build/lib.linux-armv7l-3.10/cffi
copying cffi/error.py -> build/lib.linux-armv7l-3.10/cffi
copying cffi/verifier.py -> build/lib.linux-armv7l-3.10/cffi
copying cffi/recompiler.py -> build/lib.linux-armv7l-3.10/cffi
copying cffi/api.py -> build/lib.linux-armv7l-3.10/cffi
copying cffi/vengine_cpy.py -> build/lib.linux-armv7l-3.10/cffi
copying cffi/cparser.py -> build/lib.linux-armv7l-3.10/cffi
copying cffi/ffiplatform.py -> build/lib.linux-armv7l-3.10/cffi
copying cffi/pkgconfig.py -> build/lib.linux-armv7l-3.10/cffi
copying cffi/model.py -> build/lib.linux-armv7l-3.10/cffi
copying cffi/_cffi_include.h -> build/lib.linux-armv7l-3.10/cffi
copying cffi/parse_c_type.h -> build/lib.linux-armv7l-3.10/cffi
copying cffi/_embedding.h -> build/lib.linux-armv7l-3.10/cffi
copying cffi/_cffi_errors.h -> build/lib.linux-armv7l-3.10/cffi
running build_ext
building '_cffi_backend' extension
creating build/temp.linux-armv7l-3.10
creating build/temp.linux-armv7l-3.10/c
arm-frc2022-linux-gnueabi-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -DUSE__THREAD -DHAVE_SYNC_SYNCHRONIZE -I/usr/include/ffi -I/usr/include/libffi -I/build/venv/cross/include -I/build/crosspy/include/python3.10 -c c/_cffi_backend.c -o build/temp.linux-armv7l-3.10/c/_cffi_backend.o
c/_cffi_backend.c:15:10: fatal error: ffi.h: No such file or directory
#include <ffi.h>
^~~~~~~
compilation terminated.
error: command '/usr/local/bin/arm-frc2022-linux-gnueabi-gcc' failed with exit code 1
----------------------------------------
ERROR: Command errored out with exit status 1: /build/venv/cross/bin/python3.10 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-8fd4t_d1/cffi_240eb1f0a97a43c29e5c874656e69ce2/setup.py'"'"'; __file__='"'"'/tmp/pip-install-8fd4t_d1/cffi_240eb1f0a97a43c29e5c874656e69ce2/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-j3ttetfm/install-record.txt --single-version-externally-managed --compile --install-headers /build/venv/cross/include/site/python3.10/cffi Check the logs for full command output.
WARNING: You are using pip version 21.2.4; however, version 22.0.4 is available.
You should consider upgrading via the '/build/venv/cross/bin/python3.10 -m pip install --upgrade pip' command.
I could install libffi-dev
witch provides ffi.h
iirc but same error, is it baked into the dockerfile?
Installing libffi-dev isn't going to work, because that will install for the host, not the cross-compiler environment. You would probably need to download the opkg that NI created for libffi, unpack it, and somehow point cffi at it.
It looks like you can configure cffi's ffi library via pkg-config... https://foss.heptapod.net/pypy/cffi/-/blob/branch/default/setup.py#L20.
Installing libffi-dev isn't going to work, because that will install for the host, not the cross-compiler environment. You would probably need to download the opkg that NI created for libffi, unpack it, and somehow point cffi at it.
It looks like you can configure cffi's ffi library via pkg-config... foss.heptapod.net/pypy/cffi/-/blob/branch/default/setup.py#L20.
Awesome ill look into this, i was playing around with the local docker image to see if i could get those deps on there, are pkg-config items passed from the .toml
? Sorry for the dumb questions
> CFLAGS=-I/usr/include/x86_64-linux-gnu/ pip install cffi==1.15.0
Looking in links: https://www.tortall.net/~robotpy/wheels/2022/roborio
Collecting cffi==1.15.0
Using cached cffi-1.15.0.tar.gz (484 kB)
Requirement already satisfied: pycparser in /build/venv/cross/lib/python3.10/site-packages (from cffi==1.15.0) (2.21)
Building wheels for collected packages: cffi
Building wheel for cffi (setup.py) ... error
ERROR: Command errored out with exit status 1:
command: /build/venv/cross/bin/python3.10 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-wy28lhc2/cffi_2dea19cd098c47019240ed639de2af7e/setup.py'"'"'; __file__='"'"'/tmp/pip-install-wy28lhc2/cffi_2dea19cd098c47019240ed639de2af7e/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-97ppz0a8
cwd: /tmp/pip-install-wy28lhc2/cffi_2dea19cd098c47019240ed639de2af7e/
Complete output (35 lines):
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-armv7l-3.10
creating build/lib.linux-armv7l-3.10/cffi
copying cffi/__init__.py -> build/lib.linux-armv7l-3.10/cffi
copying cffi/commontypes.py -> build/lib.linux-armv7l-3.10/cffi
copying cffi/vengine_gen.py -> build/lib.linux-armv7l-3.10/cffi
copying cffi/cffi_opcode.py -> build/lib.linux-armv7l-3.10/cffi
copying cffi/lock.py -> build/lib.linux-armv7l-3.10/cffi
copying cffi/backend_ctypes.py -> build/lib.linux-armv7l-3.10/cffi
copying cffi/setuptools_ext.py -> build/lib.linux-armv7l-3.10/cffi
copying cffi/error.py -> build/lib.linux-armv7l-3.10/cffi
copying cffi/verifier.py -> build/lib.linux-armv7l-3.10/cffi
copying cffi/recompiler.py -> build/lib.linux-armv7l-3.10/cffi
copying cffi/api.py -> build/lib.linux-armv7l-3.10/cffi
copying cffi/vengine_cpy.py -> build/lib.linux-armv7l-3.10/cffi
copying cffi/cparser.py -> build/lib.linux-armv7l-3.10/cffi
copying cffi/ffiplatform.py -> build/lib.linux-armv7l-3.10/cffi
copying cffi/pkgconfig.py -> build/lib.linux-armv7l-3.10/cffi
copying cffi/model.py -> build/lib.linux-armv7l-3.10/cffi
copying cffi/_cffi_include.h -> build/lib.linux-armv7l-3.10/cffi
copying cffi/parse_c_type.h -> build/lib.linux-armv7l-3.10/cffi
copying cffi/_embedding.h -> build/lib.linux-armv7l-3.10/cffi
copying cffi/_cffi_errors.h -> build/lib.linux-armv7l-3.10/cffi
running build_ext
building '_cffi_backend' extension
creating build/temp.linux-armv7l-3.10
creating build/temp.linux-armv7l-3.10/c
arm-frc2022-linux-gnueabi-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -I/usr/include/x86_64-linux-gnu/ -fPIC -DUSE__THREAD -DHAVE_SYNC_SYNCHRONIZE -I/usr/include/ffi -I/usr/include/libffi -I/build/venv/cross/include -I/build/crosspy/include/python3.10 -c c/_cffi_backend.c -o build/temp.linux-armv7l-3.10/c/_cffi_backend.o
arm-frc2022-linux-gnueabi-gcc -pthread -shared -I/usr/include/x86_64-linux-gnu/ build/temp.linux-armv7l-3.10/c/_cffi_backend.o -lffi -o build/lib.linux-armv7l-3.10/_cffi_backend.cpython-310-arm-linux-gnueabi.so
/usr/local/arm-frc2022-linux-gnueabi/usr/lib/gcc/arm-frc2022-linux-gnueabi/7.3.0/../../../../../../arm-frc2022-linux-gnueabi/bin/ld: cannot find -lffi
collect2: error: ld returned 1 exit status
error: command '/usr/local/bin/arm-frc2022-linux-gnueabi-gcc' failed with exit code 1
Little odd, is it looking for an lffi
artifact from the second to last command run? Im wondering if its putting that artifact somewhere outside the scope of gcc
Little odd, is it looking for an
lffi
artifact from the second to last command run? Im wondering if its putting that artifact somewhere outside the scope ofgcc
I think you need to tell it where to find libffi.so
... and it looks like using pkg-config
is the way to go there.
New plan is to use NI opkgs for cffi, start there, then figure out how to put cryptography together/use an older version of crypto (before rustc)
@virtuald is this too far behind now to continue? Should i make a new branch?
If you're still interested in it, I think starting from scratch would make sense.
Add roslibpy as a rio-wheel