build-docker fails due to apt permissions errors on arm64

[gergles]

When using main as of today (commit 1ebe42d), I can't build the container:

./utils.sh build-docker
(snipped)
Step 1/10 : FROM espressif/idf: v4.4.5
---> e2171037985f
Step 2/10 : ARG DEBIAN FRONTEND="noninteractive"
--> Using cache
- - -> 4acbb59baae1
Step 3/10 : RUN apt-get -qq update && apt-get -qq install git libusb-1.0-0 python-is-python3 python3 python3-num2words python3-pip python3-requests python3-venv sudo tio && rm -rf /var/lib/apt/lists/*
- - -> Running in d59ceea45d2a
Selecting previously unselected package python3-six.
(snipped)

Setting up python3-six (1.14.0-2) ...
[Errno 13] Permission denied: '/usr/lib/python3/dist-packages/__pycache__/six.cpython-38.pyc.281473693522384'
dpkg: error processing package python3-six (--configure):
 installed python3-six package post-installation script subprocess returned error exit status 1
(lots more like above snipped from other python packages)

This appears to be due to an issue in the upstream espressif/idf:v4.4.5 container because interactively going into that container manually and trying to install the listed packages fails. The file referenced in the apt command gets created but has a zero-byte size, so I'm not really sure what's going on here. (And obviously in the container I'm running as root, so there shouldn't be any permissions issues...)

Several of the other python3 packages complain about permissions errors in the /usr/lib/python3/dist-packages directory and subdirectories as well. I ran the destroy command (I had successfully built and installed an image about a week ago) but still receive the same error. I've purged all docker images and containers from my build host, the error persists.

The host system is arm64 Ubuntu 23.04. I tried on amd64/Debian Bookworm and it worked fine, so this may be a platform issue.

[hamishcunningham]

I think someone mentioned that 23.04 isn't supported yet? You could try 22.04 maybe?

On Fri, 30 Jun 2023, 16:03 gergles, @.***> wrote:

When using main as of today (commit 1ebe42d https://github.com/toverainc/willow/commit/1ebe42d6e2cacc2980a2986443e56973d4e5e595), I can't build the container:

./utils.sh build-docker (snipped) Step 1/10 : FROM espressif/idf: v4.4.5 ---> e2171037985f Step 2/10 : ARG DEBIAN FRONTEND="noninteractive" --> Using cache

• • -> 4acbb59baae1 Step 3/10 : RUN apt-get -qq update && apt-get -qq install git libusb-1.0-0 python-is-python3 python3 python3-num2words python3-pip python3-requests python3-venv sudo tio && rm -rf /var/lib/apt/lists/*
• • -> Running in d59ceea45d2a Selecting previously unselected package python3-six. (snipped)

Setting up python3-six (1.14.0-2) ... [Errno 13] Permission denied: '/usr/lib/python3/dist-packages/pycache/six.cpython-38.pyc.281473693522384' dpkg: error processing package python3-six (--configure): installed python3-six package post-installation script subprocess returned error exit status 1 (lots more like above snipped from other python packages)

This appears to be due to an issue in the upstream espressif/idf:v4.4.5 container because interactively going into that container manually and trying to install the listed packages fails. The file referenced in the apt command gets created but has a zero-byte size, so I'm not really sure what's going on here. (And obviously in the container I'm running as root, so there shouldn't be any permissions issues...)

Several of the other python3 packages complain about permissions errors in the /usr/lib/python3/dist-packages directory and subdirectories as well. I ran the destroy command (I had successfully built and installed an image about a week ago) but still receive the same error. I've purged all docker images and containers from my build host, the error persists.

The host system is arm64 Ubuntu 23.04. I tried on amd64/Debian Bookworm and it worked fine, so this may be a platform issue.

— Reply to this email directly, view it on GitHub https://github.com/toverainc/willow/issues/215, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABBRBFE7GEGHU4WV7M2IUPLXN3TE5ANCNFSM6AAAAAAZZ66DMQ . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[gergles]

Yes, I can confirm that 22.04 works on arm64.

Weirdly though, 23.04 did work as of a few commits ago – I had successfully built using it before. Unfortunately I blew away the environment before so can't tell you exactly when I built it, but definitely before 4ec8628 but after 02ebacc, where I do see the Dockerfile change in a566f16b6aaeeb58513fc33c0175b67d1dbe093c - where it now specified 4.4.5 instead of just 4.4, perhaps this is the change that made it not work?

Since there's a workaround to use 22.04, no big deal, but might be worth updating the readme to specifically say to use that version.

[nikito]

So the 23.04 thing I had mentioned in another thread was specific to WIS, the reason being that the nvidia docker container kit is not yet compatible with 23.04, per their documentation. Not sure this limitation applies here since I dont think willow is using a nvidia container? The difference I do see however is that you say your bookworm install was amd64, but logs show this 23.04 install is arm64. I think I remember some of the espressif libraries not supporting arm64?

Missed that last comment. Wasn't aware this worked on arm64 now. So maybe it is something with the newer ubuntu version..

[gergles]

The error output is from my arm64 machine running Ubuntu 23.04.

I was able to build successfully on an amd64 running Bookworm (no output provided, but it all worked), and on an arm64 machine running Ubuntu 22.04 (ditto).