[BUG] Failed building wheel for hexhamming breaks nextcloud

lbesnard commented 1 year ago

Is there an existing issue for this?

[X] I have searched the existing issues

Current Behavior

I've never had major issues with my nextcloud instance from linuxserver over the years, but this error got me stuck. I'm not quite sure what triggered the issue, as the last nextcloud update I performed was a few weeks ago and worked fine. Also I have watchtower running in this container, so an image from the last few weeks could have been the cause of the issue.

When starting, I noticed this message:

usermod: Failed to change ownership of the home directorychown: changing ownership of '/config/www/nextcloud/apps/recognize/node_modules/@tensorflow/tfjs-core/dist/ops/topk_test.js': Bad message

But I believe the issue is not about the ownership but about corrupted files created during the previous up -d of the container while trying to install the hexhamming python package.

I tried running a new container, with my existing /data folder, but a new empty /config folder, however the initialisation of the container always fail because of hexhamming regardless of using the old or new config folder:

Building wheels for collected packages: hexhamming
  Building wheel for hexhamming (setup.py): started
  Building wheel for hexhamming (setup.py): finished with status 'error'
  error: subprocess-exited-with-error

  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [9 lines of output]
      running bdist_wheel
      running build
      running build_ext
      building 'hexhamming' extension
      creating build
      creating build/temp.linux-x86_64-cpython-311
      creating build/temp.linux-x86_64-cpython-311/hexhamming
      gcc -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/lsiopy/include -I/usr/include/python3.11 -c hexhamming/python_hexhamming.cc -o build/temp.linux-x86_64-cpython-311/hexhamming/python_hexhamming.o -march=native
      error: command 'gcc' failed: No such file or directory
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for hexhamming
  Running setup.py clean for hexhamming
Failed to build hexhamming
ERROR: Could not build wheels for hexhamming, which is required to install pyproject.toml-based projects
[custom-init] No custom files found, skipping...

I found that this issue had somewhat similar logs,https://github.com/linuxserver/docker-nextcloud/issues/312, but I believe this must be different.

Anyway I'm pretty stuck

Expected Behavior

I can spin my nextcloud instance fine

Steps To Reproduce

Run docker-compose up -d nextcloud
check logs for failure

Environment

* OS: Ubuntu Server
* How docker service was installed: apt

CPU architecture

x86-64

Docker creation

nextcloud:
    image:  lscr.io/linuxserver/nextcloud:latest
    container_name: nextcloud
    environment:
      - WATCHTOWER_CLEANUP=true
      - PUID=$PUID_USER
      - PGID=$PGID_USER
      - TZ=$TZ
      - DOCKER_MODS=linuxserver/mods:nextcloud-mediadc|linuxserver/mods:universal-package-install
      - INSTALL_PACKAGES=ffmpeg
    labels:
      - com.centurylinklabs.watchtower.enable=true
    volumes:
      - $DOCKER_CONFIG_PATH/nextcloud:/config
      - $NEXTCLOUD_DATA_PATH:/data
    ports:
      - 9393:443
    depends_on:
      - mariadb
    restart: unless-stopped

Container logs

[mod-init] Attempting to run Docker Modification Logic
[mod-init] Adding linuxserver/mods:nextcloud-mediadc to container
[mod-init] Downloading linuxserver/mods:nextcloud-mediadc from lscr.io
[mod-init] Installing linuxserver/mods:nextcloud-mediadc
[mod-init] linuxserver/mods:nextcloud-mediadc applied to container
[mod-init] Adding linuxserver/mods:universal-package-install to container
[mod-init] Downloading linuxserver/mods:universal-package-install from lscr.io
[mod-init] Installing linuxserver/mods:universal-package-install
[mod-init] linuxserver/mods:universal-package-install applied to container
[migrations] started
[migrations] 01-nginx-site-confs-default: skipped
[migrations] 02-default-location: skipped
[migrations] done
───────────────────────────────────────

      ██╗     ███████╗██╗ ██████╗
      ██║     ██╔════╝██║██╔═══██╗
      ██║     ███████╗██║██║   ██║
      ██║     ╚════██║██║██║   ██║
      ███████╗███████║██║╚██████╔╝
      ╚══════╝╚══════╝╚═╝ ╚═════╝

   Brought to you by linuxserver.io
───────────────────────────────────────

To support LSIO projects visit:
https://www.linuxserver.io/donate/

───────────────────────────────────────
GID/UID
───────────────────────────────────────

User UID:    1000
User GID:    100
───────────────────────────────────────

using keys found in /config/keys
Initializing nextcloud 27.0.1.2 (this can take a while) ...
Setting permissions
usermod: Failed to change ownership of the home directorychown: changing ownership of '/config/www/nextcloud/apps/recognize/node_modules/@tensorflow/tfjs-core/dist/ops/topk_test.js': Bad message
**** Permissions could not be set. This is probably because your volume mounts are remote or read-only. ****
**** The app may not work properly and we will not provide support for it. ****
Initializing finished
**** The following active confs have different version dates than the samples that are shipped. ****
**** This may be due to user customization or an update to the samples. ****
**** You should compare the following files to the samples in the same folder and update them. ****
**** Use the link at the top of the file to view the changelog. ****
┌────────────┬────────────┬────────────────────────────────────────────────────────────────────────┐
│  old date  │  new date  │ path                                                                   │
├────────────┼────────────┼────────────────────────────────────────────────────────────────────────┤
│ 2022-08-20 │ 2023-06-24 │ /config/nginx/ssl.conf                                                 │
│ 2018-08-16 │ 2023-04-13 │ /config/nginx/nginx.conf                                               │
└────────────┴────────────┴────────────────────────────────────────────────────────────────────────┘
**** Adding ffmpeg to OS package install list ****
**** Adding pillow-heif, hexhamming, pywavelets and their deps to package install lists ****
[mod-init] **** Installing all mod packages ****
fetch http://dl-cdn.alpinelinux.org/alpine/v3.18/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.18/community/x86_64/APKINDEX.tar.gz
(1/52) Installing mpdecimal (2.5.1-r2)
(2/52) Installing python3 (3.11.4-r0)
(3/52) Installing python3-pycache-pyc0 (3.11.4-r0)
(4/52) Installing pyc (0.1-r0)
(5/52) Installing py3-asn1crypto-pyc (1.5.1-r2)
(6/52) Installing python3-pyc (3.11.4-r0)
(7/52) Installing py3-asn1crypto (1.5.1-r2)
(8/52) Installing py3-cparser (2.21-r2)
(9/52) Installing py3-cparser-pyc (2.21-r2)
(10/52) Installing py3-cffi (1.15.1-r3)
(11/52) Installing py3-cffi-pyc (1.15.1-r3)
(12/52) Installing py3-cryptography (40.0.2-r1)
(13/52) Installing py3-cryptography-pyc (40.0.2-r1)
(14/52) Installing libquadmath (12.2.1_git20220924-r10)
(15/52) Installing libgfortran (12.2.1_git20220924-r10)
(16/52) Installing openblas (0.3.23-r0)
(17/52) Installing py3-numpy (1.24.4-r0)
(18/52) Installing py3-numpy-pyc (1.24.4-r0)
(19/52) Installing libimagequant (4.2.0-r0)
(20/52) Installing openjpeg (2.5.0-r3)
(21/52) Installing py3-pillow (9.5.0-r1)
(22/52) Installing py3-pillow-pyc (9.5.0-r1)
(23/52) Installing py3-parsing (3.0.9-r2)
(24/52) Installing py3-parsing-pyc (3.0.9-r2)
(25/52) Installing py3-packaging (23.1-r1)
(26/52) Installing py3-packaging-pyc (23.1-r1)
(27/52) Installing py3-setuptools (67.7.2-r0)
(28/52) Installing py3-setuptools-pyc (67.7.2-r0)
(29/52) Installing py3-pip (23.1.2-r0)
(30/52) Installing py3-pip-pyc (23.1.2-r0)
(31/52) Installing py3-pynacl (1.5.0-r4)
(32/52) Installing py3-pynacl-pyc (1.5.0-r4)
(33/52) Installing py3-platformdirs (3.5.0-r0)
(34/52) Installing py3-platformdirs-pyc (3.5.0-r0)
(35/52) Installing py3-certifi (2023.5.7-r0)
(36/52) Installing py3-certifi-pyc (2023.5.7-r0)
(37/52) Installing py3-charset-normalizer (3.1.0-r1)
(38/52) Installing py3-charset-normalizer-pyc (3.1.0-r1)
(39/52) Installing py3-idna (3.4-r4)
(40/52) Installing py3-idna-pyc (3.4-r4)
(41/52) Installing py3-urllib3 (1.26.15-r1)
(42/52) Installing py3-urllib3-pyc (1.26.15-r1)
(43/52) Installing py3-requests (2.31.0-r0)
(44/52) Installing py3-requests-pyc (2.31.0-r0)
(45/52) Installing py3-pooch (1.7.0-r1)
(46/52) Installing py3-pooch-pyc (1.7.0-r1)
(47/52) Installing python3-dev (3.11.4-r0)
(48/52) Installing py3-numpy-f2py (1.24.4-r0)
(49/52) Installing py3-scipy (1.10.1-r2)
(50/52) Installing py3-scipy-pyc (1.10.1-r2)
(51/52) Installing supervisor (4.2.5-r2)
(52/52) Installing supervisor-pyc (4.2.5-r2)
Executing busybox-1.36.1-r1.trigger
OK: 609 MiB in 310 packages
[mod-init] **** Installing all pip packages ****
**** Creating venv ****
Requirement already satisfied: pip in /lsiopy/lib/python3.11/site-packages (23.1.2)
Collecting pip
  Downloading pip-23.2.1-py3-none-any.whl (2.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.1/2.1 MB 5.4 MB/s eta 0:00:00
Collecting wheel
  Downloading wheel-0.41.0-py3-none-any.whl (64 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 64.7/64.7 kB 1.3 MB/s eta 0:00:00
Requirement already satisfied: setuptools in /lsiopy/lib/python3.11/site-packages (65.5.0)
Collecting setuptools
  Downloading setuptools-68.0.0-py3-none-any.whl (804 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 804.0/804.0 kB 4.5 MB/s eta 0:00:00
Installing collected packages: wheel, setuptools, pip
  Attempting uninstall: setuptools
    Found existing installation: setuptools 65.5.0
    Uninstalling setuptools-65.5.0:
      Successfully uninstalled setuptools-65.5.0
  Attempting uninstall: pip
    Found existing installation: pip 23.1.2
    Uninstalling pip-23.1.2:
      Successfully uninstalled pip-23.1.2
Successfully installed pip-23.2.1 setuptools-68.0.0 wheel-0.41.0
Looking in links: https://wheel-index.linuxserver.io/alpine-3.18/
Collecting pillow-heif
  Obtaining dependency information for pillow-heif from https://files.pythonhosted.org/packages/f9/0c/26d9257be4aa64185dd2f750712b69a237e49cbf45ea0c0be8d251fe004b/pillow_heif-0.12.0-cp311-cp311-musllinux_1_1_x86_64.whl.metadata
  Downloading pillow_heif-0.12.0-cp311-cp311-musllinux_1_1_x86_64.whl.metadata (9.1 kB)
Collecting hexhamming
  Downloading hexhamming-2.2.3.tar.gz (15 kB)
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Collecting pywavelets
  Downloading https://wheels.linuxserver.io/alpine-3.18/PyWavelets-1.4.1-cp311-cp311-linux_x86_64.whl (7.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7.0/7.0 MB 683.4 kB/s eta 0:00:00
Collecting pillow>=8.4.0 (from pillow-heif)
  Obtaining dependency information for pillow>=8.4.0 from https://files.pythonhosted.org/packages/40/58/0a62422b3cf188dac72fe6c54b6f3f372ec2e84043eb4f8d2158626992b7/Pillow-10.0.0-cp311-cp311-musllinux_1_1_x86_64.whl.metadata
  Downloading Pillow-10.0.0-cp311-cp311-musllinux_1_1_x86_64.whl.metadata (9.5 kB)
Collecting numpy>=1.17.3 (from pywavelets)
  Obtaining dependency information for numpy>=1.17.3 from https://files.pythonhosted.org/packages/55/ee/a9cb689bab52cb617094148a2db7f87f89fb390682392545e8751a9cf06b/numpy-1.25.1-cp311-cp311-musllinux_1_1_x86_64.whl.metadata
  Downloading numpy-1.25.1-cp311-cp311-musllinux_1_1_x86_64.whl.metadata (5.6 kB)
Downloading pillow_heif-0.12.0-cp311-cp311-musllinux_1_1_x86_64.whl (7.9 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7.9/7.9 MB 6.2 MB/s eta 0:00:00
Downloading numpy-1.25.1-cp311-cp311-musllinux_1_1_x86_64.whl (17.5 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 17.5/17.5 MB 6.1 MB/s eta 0:00:00
Downloading Pillow-10.0.0-cp311-cp311-musllinux_1_1_x86_64.whl (3.4 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.4/3.4 MB 6.2 MB/s eta 0:00:00
Building wheels for collected packages: hexhamming
  Building wheel for hexhamming (setup.py): started
  Building wheel for hexhamming (setup.py): finished with status 'error'
  error: subprocess-exited-with-error

  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [9 lines of output]
      running bdist_wheel
      running build
      running build_ext
      building 'hexhamming' extension
      creating build
      creating build/temp.linux-x86_64-cpython-311
      creating build/temp.linux-x86_64-cpython-311/hexhamming
      gcc -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/lsiopy/include -I/usr/include/python3.11 -c hexhamming/python_hexhamming.cc -o build/temp.linux-x86_64-cpython-311/hexhamming/python_hexhamming.o -march=native
      error: command 'gcc' failed: No such file or directory
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for hexhamming
  Running setup.py clean for hexhamming
Failed to build hexhamming
ERROR: Could not build wheels for hexhamming, which is required to install pyproject.toml-based projects
[custom-init] No custom files found, skipping...
[ls.io-init] done.

github-actions[bot] commented 1 year ago

Thanks for opening your first issue here! Be sure to follow the relevant issue templates, or risk having this issue marked as invalid.

lbesnard commented 1 year ago

I'd like to add that even using the previous image nextcloud:27.0.1-ls258 leads to the same problem with hexhamming

aptalca commented 1 year ago

@lbesnard It looks like hexhamming was last released a year and a half ago, which was before the python 3.11 releases. Therefore pypi does not contain the wheels for 3.11, which nextcloud got rebased on recently.

I just pushed the wheels to our own repo so it should work now. I also have an update for the mod to fix some of the optional dependencies.

If you can test this PR mod and let us know if it works, it would be much appreciated: ghcr.io/linuxserver/mods:pull_request_733

You would replace linuxserver/mods:nextcloud-mediadc with ghcr.io/linuxserver/mods:pull_request_733 in the DOCKER_MODS variable to test.

Thanks

PR for reference: https://github.com/linuxserver/docker-mods/pull/733

lbesnard commented 1 year ago

Perfect! Thanks a lot for such a quick fix. My instance works as expected.

Cheers

linuxserver / docker-nextcloud