search

abstractfoundry / lumicube-daemon

10 stars 13 forks source link

Exception when running Lava Lamp module in Ubuntu 22.04.1 LTS (Pi 4B) #4

Closed mfoo closed 2 years ago

mfoo commented 2 years ago

Hello!

I've now got the Lumicube running after building with Maven using a new flashed Ubuntu Server 22.04.1 LTS image. I had some trouble with install.py (for another issue when I have time!) and so had to run locally instead, meaning I had to install a bunch of dependencies manually. I'm hoping that this is the cause of the issue.

The issue exists only with the Lava Lamp example. I can see the following exception in the log coming from Python:

Traceback (most recent call last):
  File "main", line 15, in paint_cube
  File "main", line 6, in lava_colour
  File "/tmp/foundry-script-context-8120301071552718362/foundry_api/standard_library.py", line 371, in noise4
    self._ensure_ffi_connected()
  File "/tmp/foundry-script-context-8120301071552718362/foundry_api/standard_library.py", line 360, in _ensure_ffi_connected
    self._connect_ffi()
  File "/tmp/foundry-script-context-8120301071552718362/foundry_api/standard_library.py", line 341, in _connect_ffi
    lib = ffi.dlopen(_script_context + '/noise/open-simplex-noise-' + variant + '.so')
  File "/home/ubuntu/.local/lib/python3.10/site-packages/cffi/api.py", line 150, in dlopen
    lib, function_cache = _make_ffi_library(self, name, flags)
  File "/home/ubuntu/.local/lib/python3.10/site-packages/cffi/api.py", line 832, in _make_ffi_library
    backendlib = _load_backend_lib(backend, libname, flags)
  File "/home/ubuntu/.local/lib/python3.10/site-packages/cffi/api.py", line 827, in _load_backend_lib
    raise OSError(msg)
  OSError: cannot load library '/tmp/foundry-script-context-8120301071552718362/noise/open-simplex-noise-x64.so': /tmp/foundry-script-context-8120301071552718362/noise/open-simplex-noise-x64.so: cannot open shared object file: No such file or directory.  Additionally, ctypes.util.find_library() did not manage to locate a library called '/tmp/foundry-script-context-8120301071552718362/noise/open-simplex-noise-x64.so'

However I can see that the file does exist and is a 64 bit dynamically linked shared object file compiled for x86-64, but ldd is having problems showing the contents (maybe this is because it's statically linked and the file output saying "dynamically linked" is incorrect?):

ubuntu@ubuntu:~/repositories/lumicube-daemon$ file /tmp/foundry-script-context-8120301071552718362/noise/open-simplex-noise-*.so
/tmp/foundry-script-context-8120301071552718362/noise/open-simplex-noise-arm.so: ELF 32-bit LSB shared object, ARM, EABI5 version 1 (SYSV), dynamically linked, BuildID[sha1]=0ccf640c06db594955478859d4ef416aa908bc49, not stripped
/tmp/foundry-script-context-8120301071552718362/noise/open-simplex-noise-x64.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=e39ad4b807f46196fc01cc5636700ac54bb9d98e, not stripped

ubuntu@ubuntu:~/repositories/lumicube-daemon$ ldd /tmp/foundry-script-context-8120301071552718362/noise/open-simplex-noise-*.so
/tmp/foundry-script-context-8120301071552718362/noise/open-simplex-noise-arm.so:
    not a dynamic executable
/tmp/foundry-script-context-8120301071552718362/noise/open-simplex-noise-x64.so:
    not a dynamic executable

Either way, I definitely can't load it:

ubuntu@ubuntu:~/repositories/lumicube-daemon$ python3
Python 3.10.6 (main, Aug 10 2022, 11:40:04) [GCC 11.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from os.path import exists
>>> exists('/tmp/foundry-script-context-8120301071552718362/noise/open-simplex-noise-x64.so')
True
>>> from cffi import FFI
>>> ffi = FFI()
>>> ffi.dlopen('/tmp/foundry-script-context-8120301071552718362/noise/open-simplex-noise-x64.so')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/ubuntu/.local/lib/python3.10/site-packages/cffi/api.py", line 150, in dlopen
    lib, function_cache = _make_ffi_library(self, name, flags)
  File "/home/ubuntu/.local/lib/python3.10/site-packages/cffi/api.py", line 832, in _make_ffi_library
    backendlib = _load_backend_lib(backend, libname, flags)
  File "/home/ubuntu/.local/lib/python3.10/site-packages/cffi/api.py", line 827, in _load_backend_lib
    raise OSError(msg)
OSError: cannot load library '/tmp/foundry-script-context-8120301071552718362/noise/open-simplex-noise-x64.so': /tmp/foundry-script-context-8120301071552718362/noise/open-simplex-noise-x64.so: cannot open shared object file: No such file or directory.  Additionally, ctypes.util.find_library() did not manage to locate a library called '/tmp/foundry-script-context-8120301071552718362/noise/open-simplex-noise-x64.so'

I've attached the readelf output if that's useful:

ubuntu@ubuntu:~/repositories/lumicube-daemon$ readelf -l /tmp/foundry-script-context-8120301071552718362/noise/open-simplex-noise-x64.so

Elf file type is DYN (Shared object file)
Entry point 0x1070
There are 9 program headers, starting at offset 64

Program Headers:
  Type           Offset             VirtAddr           PhysAddr
                 FileSiz            MemSiz              Flags  Align
  LOAD           0x0000000000000000 0x0000000000000000 0x0000000000000000
                 0x0000000000000638 0x0000000000000638  R      0x1000
  LOAD           0x0000000000001000 0x0000000000001000 0x0000000000001000
                 0x0000000000005cb1 0x0000000000005cb1  R E    0x1000
  LOAD           0x0000000000007000 0x0000000000007000 0x0000000000007000
                 0x000000000000046c 0x000000000000046c  R      0x1000
  LOAD           0x0000000000007e10 0x0000000000008e10 0x0000000000008e10
                 0x0000000000000228 0x0000000000000230  RW     0x1000
  DYNAMIC        0x0000000000007e20 0x0000000000008e20 0x0000000000008e20
                 0x00000000000001c0 0x00000000000001c0  RW     0x8
  NOTE           0x0000000000000238 0x0000000000000238 0x0000000000000238
                 0x0000000000000024 0x0000000000000024  R      0x4
  GNU_EH_FRAME   0x00000000000071f0 0x00000000000071f0 0x00000000000071f0
                 0x0000000000000064 0x0000000000000064  R      0x4
  GNU_STACK      0x0000000000000000 0x0000000000000000 0x0000000000000000
                 0x0000000000000000 0x0000000000000000  RW     0x10
  GNU_RELRO      0x0000000000007e10 0x0000000000008e10 0x0000000000008e10
                 0x00000000000001f0 0x00000000000001f0  R      0x1

 Section to Segment mapping:
  Segment Sections...
   00     .note.gnu.build-id .gnu.hash .dynsym .dynstr .gnu.version .gnu.version_r .rela.dyn .rela.plt 
   01     .init .plt .plt.got .text .fini 
   02     .rodata .eh_frame_hdr .eh_frame 
   03     .init_array .fini_array .dynamic .got .got.plt .data .bss 
   04     .dynamic 
   05     .note.gnu.build-id 
   06     .eh_frame_hdr 
   07     
   08     .init_array .fini_array .dynamic .got

Any help would be appreciated!

mfoo commented 2 years ago

Looks like https://github.com/smcameron/open-simplex-noise-in-c is being used - would it be simpler to use a Python implementation? E.g. https://pypi.org/project/opensimplex/ is MIT licensed so should be fine to include.

abstractfoundry commented 2 years ago

Hi @mfoo - we were originally using https://pypi.org/project/opensimplex/ - but it's so slow on the Raspberry Pi, it causes animations to look quite jerky! I presume you're running a 64-bit Ubuntu distribution on your Raspberry Pi (which we don't officially support, just the 32-bit Raspberry Pi OS desktop version). That said you should be able to get our software on Ubuntu (at least it's Debian based, so not too dissimilar). Basically you're going to need to build an aarch64 version of the .so file (as the x64 library you found is for amd64) and get Python to try linking to your own version.

To build the library for aarch64. first apt install docker qemu qemu-user-static binfmt-support and then run the following script:

cat > Dockerfile <<\EOF

FROM debian:buster-20210111

RUN apt update && apt install -y git build-essential

RUN git clone https://github.com/smcameron/open-simplex-noise-in-c /root/OpenSimplex
WORKDIR /root/OpenSimplex
RUN gcc -O2 -fPIC -c -o open-simplex-noise.o open-simplex-noise.c
RUN gcc -O2 -shared -o open-simplex-noise.so open-simplex-noise.o

EOF

docker image rm debian:buster-20210111 # Work around Docker not supporting several platform-specific versions of the same image.
docker build --platform linux/arm64/v8 -t build-simplex-aarch64 .
docker run --rm --entrypoint cat build-simplex-aarch64 /root/OpenSimplex/open-simplex-noise.so > open-simplex-noise-aarch64.so
chmod +x open-simplex-noise-aarch64.so

This script successfully generated a file when I just ran it, but I haven't actually tested that it's possible to link to. You can package the resulting .so next to the other versions in the daemon. Finally _platform_variant in standard_library.py needs to be modified to detect and return 'aarch64' so the correct .so is linked. Hope that's some help!

mfoo commented 2 years ago

Worked first time, thanks!

I'll have a think about this, maybe there's some improvement we can make here that doesn't involve including multiple .so binaries in the repo!