robbiewu008 commented 3 years ago

robbiewu@robbiedeair seL4-CAmkES-L4v-dockerfiles % make user
scripts/utils/check_for_old_docker_imgs.sh
WARNING: Unable to check if your trustworthysystems docker images are getting a bit old!
         The date command did not behave as expected. Skipping the check.
docker build --force-rm=true \
        --build-arg=USER_BASE_IMG=trustworthysystems/camkes \
        -f dockerfiles/extras.Dockerfile \
        -t extras \
        .
[+] Building 276.2s (7/7) FINISHED                                                        
 => [internal] load build definition from extras.Dockerfile                          0.0s
 => => transferring dockerfile: 607B                                                 0.0s
 => [internal] load .dockerignore                                                    0.0s
 => => transferring context: 153B                                                    0.0s
 => [internal] load metadata for docker.io/trustworthysystems/camkes:latest          4.2s
 => [auth] trustworthysystems/camkes:pull token for registry-1.docker.io             0.0s
 => [1/2] FROM docker.io/trustworthysystems/camkes@sha256:2dfe8e125a61b661e475a75  121.6s
 => => resolve docker.io/trustworthysystems/camkes@sha256:2dfe8e125a61b661e475a753f  0.0s
 => => sha256:2dfe8e125a61b661e475a753f6081dc12c95a63c245f8eb56fe58 1.79kB / 1.79kB  0.0s
 => => sha256:dd50eddb09bea873c9139295bdf5e18c0cea3a276d63f5edf1086 8.26kB / 8.26kB  0.0s
 => => sha256:6fc55abfba0d87027dac918d6beba1579daefa4fa39e1a1f8c3 12.13kB / 12.13kB  1.1s
 => => sha256:afb6ec6fdc1c3ba04f7a56db32c5ff5ff38962dc4cd0ffdef5b 27.10MB / 27.10MB  1.2s
 => => sha256:03d756ce13c072cc33240799e524316d1942d0f8f21e51bd 132.98MB / 132.98MB  27.0s
 => => sha256:cfaf6fd66bfefddf974a6f11ef5f94f7e588f312812050c54de 12.10kB / 12.10kB  1.4s
 => => sha256:4092e53e147f12f08a54990ea088cf98d448610536f70870 702.53MB / 702.53MB  91.3s
 => => extracting sha256:afb6ec6fdc1c3ba04f7a56db32c5ff5ff38962dc4cd0ffdef5beaa0ce2  0.8s
 => => sha256:815c86a1454a9f90ffe7848d985c17648ac3cda2ee0baf0e3dc 12.10kB / 12.10kB  1.8s
 => => sha256:bb3669078ee93402dccc7d8e61182849fac976009fa04c66 476.80MB / 476.80MB  76.2s
 => => extracting sha256:6fc55abfba0d87027dac918d6beba1579daefa4fa39e1a1f8c3d98bac3  0.0s
 => => extracting sha256:03d756ce13c072cc33240799e524316d1942d0f8f21e51bdc91901c764  3.1s
 => => extracting sha256:cfaf6fd66bfefddf974a6f11ef5f94f7e588f312812050c54de64c206f  0.0s
 => => extracting sha256:4092e53e147f12f08a54990ea088cf98d448610536f7087005294f59b  17.8s
 => => extracting sha256:815c86a1454a9f90ffe7848d985c17648ac3cda2ee0baf0e3dc54838d1  0.0s
 => => extracting sha256:bb3669078ee93402dccc7d8e61182849fac976009fa04c66c0f836b98  11.6s
 => [2/2] RUN apt-get update -q     && apt-get install -y --no-install-recommends  150.0s
 => exporting to image                                                               0.3s
 => => exporting layers                                                              0.3s
 => => writing image sha256:d488115963071b0845c2621f3667b1b56e68b9a2f07c6236e79bc19  0.0s 
 => => naming to docker.io/library/extras                                            0.0s 

Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix them
docker build --force-rm=true \
        --build-arg=EXTRAS_IMG=extras \
        --build-arg=UNAME=robbiewu \
        --build-arg=UID=501 \
        --build-arg=GID=20 \
        --build-arg=GROUP=staff \
        -f dockerfiles/user.Dockerfile \
        -t user_img-robbiewu .
[+] Building 3.0s (4/4) FINISHED                                                          
 => [internal] load build definition from user.Dockerfile                            0.0s
 => => transferring dockerfile: 358B                                                 0.0s
 => [internal] load .dockerignore                                                    0.0s
 => => transferring context: 34B                                                     0.0s
 => ERROR [internal] load metadata for docker.io/library/extras:latest               2.9s
 => [auth] library/extras:pull token for registry-1.docker.io                        0.0s
------
 > [internal] load metadata for docker.io/library/extras:latest:
------
failed to solve with frontend dockerfile.v0: failed to create LLB definition: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed
make: *** [build_user] Error 1
robbiewu@robbiedeair seL4-CAmkES-L4v-dockerfiles % docker images -a
REPOSITORY   TAG       IMAGE ID       CREATED          SIZE
extras       latest    d48811596307   54 seconds ago   4.92GB

lsf37 commented 3 years ago

This looks like your user does not have permission to build docker images. It might have to be in the docker group to do that, depending on your system and setup.

Have you been able to build/run any other docker images like the ones from the getting-started guide at docker?

robbiewu008 commented 3 years ago

This looks like your user does not have permission to build docker images. It might have to be in the docker group to do that, depending on your system and setup.

Have you been able to build/run any other docker images like the ones from the getting-started guide at docker?

It's OK with my docker environment.

robbie.wu@MBFVFG30ELQ6LW seL4-CAmkES-L4v-dockerfiles % docker run -d -p 80:80 docker/getting-started Unable to find image 'docker/getting-started:latest' locally latest: Pulling from docker/getting-started 595b0fe564bb: Pull complete 31e3f3692bd6: Pull complete ba4d371dcb40: Pull complete efb7a2f13438: Pull complete 3d4e2f872060: Pull complete 43416c5e3473: Pull complete fd101b99d290: Pull complete 87904de4d38c: Pull complete Digest: sha256:10555bb0c50e13fc4dd965ddb5f00e948ffa53c13ff15dcdc85b7ab65e1f240b Status: Downloaded newer image for docker/getting-started:latest 69f261b8377d2a529bdf929e4efafc801357f7c018e0e31bb44efd44b10a2dbe robbie.wu@MBFVFG30ELQ6LW seL4-CAmkES-L4v-dockerfiles % docker images -a REPOSITORY TAG IMAGE ID CREATED SIZE docker/getting-started latest 613921574f76 3 months ago 26.7MB robbie.wu@MBFVFG30ELQ6LW seL4-CAmkES-L4v-dockerfiles % docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 69f261b8377d docker/getting-started "/docker-entrypoint.…" 16 seconds ago Up 15 seconds 0.0.0.0:80->80/tcp, :::80->80/tcp ecstatic_lederberg

robbiewu008 commented 3 years ago

This looks like your user does not have permission to build docker images. It might have to be in the docker group to do that, depending on your system and setup.

Have you been able to build/run any other docker images like the ones from the getting-started guide at docker?

It's also OK to build a getting-started images:

robbie.wu@MBFVFG30ELQ6LW GettingStarted % vscode app robbie.wu@MBFVFG30ELQ6LW GettingStarted % cd app robbie.wu@MBFVFG30ELQ6LW app % ls Dockerfile package.json spec src yarn.lock robbie.wu@MBFVFG30ELQ6LW app % docker build -t getting-started . [+] Building 122.7s (11/11) FINISHED
=> [internal] load build definition from Dockerfile 0.0s => => transferring dockerfile: 44B 0.0s => [internal] load .dockerignore 0.0s => => transferring context: 2B 0.0s => [internal] load metadata for docker.io/library/node:12-alpine 2.7s => [auth] library/node:pull token for registry-1.docker.io 0.0s => [internal] load build context 0.0s => => transferring context: 6.58kB 0.0s => CACHED [1/5] FROM docker.io/library/node:12-alpine@sha256:1ea5900145028957ec0e7 0.0s => [2/5] RUN apk add --no-cache python g++ make 5.7s => [3/5] WORKDIR /app 0.0s => [4/5] COPY . . 0.0s => [5/5] RUN yarn install --production 112.9s => exporting to image 1.3s => => exporting layers 1.3s => => writing image sha256:56e455d837e9334fe0bd8865c16e5aff127da5b2721b2572f6222eb 0.0s => => naming to docker.io/library/getting-started 0.0s

Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix them robbie.wu@MBFVFG30ELQ6LW app % docker scan Docker Scan relies upon access to Snyk, a third party provider, do you consent to proceed using Snyk? (y/N) y Usage: docker scan [OPTIONS] IMAGE

A tool to scan your images

Options: --accept-license Accept using a third party scanning provider --dependency-tree Show dependency tree with scan results --exclude-base Exclude base image from vulnerability scanning (requires --file) -f, --file string Dockerfile associated with image, provides more detailed results --group-issues Aggregate duplicated vulnerabilities and group them to a single one (requires --json) --json Output results in JSON format --login Authenticate to the scan provider using an optional token (with --token), or web base token if empty --reject-license Reject using a third party scanning provider --severity string Only report vulnerabilities of provided level or higher (low|medium|high) --token string Authentication token to login to the third party scanning provider --version Display version of the scan plugin "docker scan" requires exactly 1 argument robbie.wu@MBFVFG30ELQ6LW app % docker scan getting-started

Testing getting-started...

Package manager: apk Project name: docker-image|getting-started Docker image: getting-started Platform: linux/arm64

✓ Tested 38 dependencies for known vulnerabilities, no vulnerable paths found.

For more free scans that keep your images secure, sign up to Snyk at https://dockr.ly/3ePqVcp

lsf37 commented 3 years ago

Interesting. Does docker run -ti --entrypoint bash trustworthysystems/sel4 work for you?

lsf37 commented 3 years ago

Ok, I was able to reproduce the problem, and it appears specific to docker buildkit on the M1.

The following worked for me as a workaround:

export DOCKER_BUILDLOT=0

(and then make user as before)

lsf37 commented 3 years ago

I'm not sure how well docker works yet on the M1, at least when running x86 containers. Last time I checked, support for that was experimental and I had plenty of crashes from docker when I tried, but there is hope that things have improved significantly in since then.

robbiewu008 commented 3 years ago

docker run -ti --entrypoint bash trustworthysystems/sel4

It does NOT work for me.

robbie.wu@MBFVFG30ELQ6LW seL4-CAmkES-L4v-dockerfiles % docker run -ti --entrypoint bash trustworthysystems/sel4 Unable to find image 'trustworthysystems/sel4:latest' locally latest: Pulling from trustworthysystems/sel4 afb6ec6fdc1c: Already exists 322dc1ce076f: Already exists caf6234671a5: Already exists fed4b0e4768f: Already exists 7a5b7604e264: Already exists Digest: sha256:18d32c7154a55590625a36ce65221df46bad22b8426f899dcc72e683f46b1cf6 Status: Downloaded newer image for trustworthysystems/sel4:latest WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested

robbiewu008 commented 3 years ago

export DOCKER_BUILDLOT=0

it does NOT work for me too.

robbie.wu@MBFVFG30ELQ6LW seL4-CAmkES-L4v-dockerfiles % export DOCKER_BUILDLOT=0 robbie.wu@MBFVFG30ELQ6LW seL4-CAmkES-L4v-dockerfiles % make user scripts/utils/check_for_old_docker_imgs.sh WARNING: Unable to check if your trustworthysystems docker images are getting a bit old! The date command did not behave as expected. Skipping the check. docker build --force-rm=true \ --build-arg=USER_BASE_IMG=trustworthysystems/camkes \ -f dockerfiles/extras.Dockerfile \ -t extras \ . [+] Building 3.2s (7/7) FINISHED
=> [internal] load build definition from extras.Dockerfile 0.0s => => transferring dockerfile: 614B 0.0s => [internal] load .dockerignore 0.0s => => transferring context: 153B 0.0s => [internal] load metadata for docker.io/trustworthysystems/camkes:latest 3.1s => [auth] trustworthysystems/camkes:pull token for registry-1.docker.io 0.0s => [1/2] FROM docker.io/trustworthysystems/camkes@sha256:2dfe8e125a61b661e475a753f 0.0s => CACHED [2/2] RUN apt-get update -q && apt-get install -y --no-install-recom 0.0s => exporting to image 0.0s => => exporting layers 0.0s => => writing image sha256:696d2365aca9d2894b6281cbf771a1a170632dc71863809228ff1de 0.0s => => naming to docker.io/library/extras 0.0s docker build --force-rm=true \ --build-arg=EXTRAS_IMG=extras \ --build-arg=UNAME=robbie.wu \ --build-arg=UID=701124824 \ --build-arg=GID=656741719 \ --build-arg=GROUP=656741719 \ -f dockerfiles/user.Dockerfile \ -t user_img-robbie.wu . [+] Building 3.0s (4/4) FINISHED
=> [internal] load build definition from user.Dockerfile 0.0s => => transferring dockerfile: 365B 0.0s => [internal] load .dockerignore 0.0s => => transferring context: 34B 0.0s => ERROR [internal] load metadata for docker.io/library/extras:latest 2.9s => [auth] library/extras:pull token for registry-1.docker.io 0.0s

[internal] load metadata for docker.io/library/extras:latest:

failed to solve with frontend dockerfile.v0: failed to create LLB definition: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed make: *** [build_user] Error 1

lsf37 commented 3 years ago

Sorry, autocorrect typo. It should be

export DOCKER_BUILDKIT=0

lsf37 commented 3 years ago

docker run -ti --entrypoint bash trustworthysystems/sel4

It does NOT work for me.

robbie.wu@MBFVFG30ELQ6LW seL4-CAmkES-L4v-dockerfiles % docker run -ti --entrypoint bash trustworthysystems/sel4
Unable to find image 'trustworthysystems/sel4:latest' locally
latest: Pulling from trustworthysystems/sel4
afb6ec6fdc1c: Already exists
322dc1ce076f: Already exists
caf6234671a5: Already exists
fed4b0e4768f: Already exists
7a5b7604e264: Already exists
Digest: sha256:18d32c7154a55590625a36ce65221df46bad22b8426f899dcc72e683f46b1cf6
Status: Downloaded newer image for trustworthysystems/sel4:latest
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested

So this one looks successful to me, but the deciding bit would be the next line after this. For me it is:

root@...<image hex id>...:/#

i.e. if you see that, at that point you are inside the docker container.

robbiewu008 commented 3 years ago

export DOCKER_BUILDKIT=0

still does not work.

robbie.wu@MBFVFG30ELQ6LW seL4-CAmkES-L4v-dockerfiles % export DOCKER_BUILDKIT=0 robbie.wu@MBFVFG30ELQ6LW seL4-CAmkES-L4v-dockerfiles % make user scripts/utils/check_for_old_docker_imgs.sh WARNING: Unable to check if your trustworthysystems docker images are getting a bit old! The date command did not behave as expected. Skipping the check. docker build --force-rm=true \ --build-arg=USER_BASE_IMG=trustworthysystems/camkes \ -f dockerfiles/extras.Dockerfile \ -t extras \ . Sending build context to Docker daemon 101.9kB Step 1/3 : ARG USER_BASE_IMG=trustworthysystems/sel4 Step 2/3 : FROM $USER_BASE_IMG latest: Pulling from trustworthysystems/camkes afb6ec6fdc1c: Pull complete 6fc55abfba0d: Pull complete 03d756ce13c0: Pull complete cfaf6fd66bfe: Pull complete 4092e53e147f: Pull complete 815c86a1454a: Pull complete bb3669078ee9: Pull complete Digest: sha256:2dfe8e125a61b661e475a753f6081dc12c95a63c245f8eb56fe580e3c80eedca Status: Downloaded newer image for trustworthysystems/camkes:latest ---> dd50eddb09be Step 3/3 : RUN apt-get update -q && apt-get install -y --no-install-recommends cowsay sudo ---> [Warning] The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested ---> Running in 465908e3f8ee Get:1 http://security.debian.org/debian-security buster/updates InRelease [65.4 kB] Get:2 http://deb.debian.org/debian buster InRelease [122 kB] Get:3 http://deb.debian.org/debian buster-updates InRelease [51.9 kB] Ign:4 http://deb.debian.org/debian stretch InRelease Get:5 http://deb.debian.org/debian bullseye InRelease [113 kB] Get:6 http://security.debian.org/debian-security buster/updates/main i386 Packages [302 kB] Get:7 http://deb.debian.org/debian stretch Release [118 kB] Get:8 http://security.debian.org/debian-security buster/updates/main amd64 Packages [303 kB] Get:9 http://deb.debian.org/debian stretch Release.gpg [3,177 B] Get:10 http://security.debian.org/debian-security buster/updates/main armel Packages [277 kB] Get:11 http://security.debian.org/debian-security buster/updates/main armhf Packages [297 kB] Get:12 http://deb.debian.org/debian buster/main armel Packages [7,628 kB] Get:13 http://deb.debian.org/debian buster/main armhf Packages [7,698 kB] Get:14 http://deb.debian.org/debian buster/main i386 Packages [7,863 kB] Get:15 http://deb.debian.org/debian buster/main amd64 Packages [7,907 kB] Get:16 http://deb.debian.org/debian buster-updates/main armel Packages [14.1 kB] Get:17 http://deb.debian.org/debian buster-updates/main i386 Packages [15.2 kB] Get:18 http://deb.debian.org/debian buster-updates/main armhf Packages [14.4 kB] Get:19 http://deb.debian.org/debian buster-updates/main amd64 Packages [15.2 kB] Get:20 http://deb.debian.org/debian bullseye/main armel Packages [7,861 kB] Get:21 http://deb.debian.org/debian bullseye/main i386 Packages [8,117 kB] Get:22 http://deb.debian.org/debian bullseye/main amd64 Packages [8,178 kB] Get:23 http://deb.debian.org/debian bullseye/main armhf Packages [7,941 kB] Get:24 http://deb.debian.org/debian stretch/main armel Packages [6,879 kB] Get:25 http://deb.debian.org/debian stretch/main armhf Packages [6,908 kB] Get:26 http://deb.debian.org/debian stretch/main amd64 Packages [7,080 kB] Get:27 http://deb.debian.org/debian stretch/main i386 Packages [7,059 kB] Fetched 92.8 MB in 53s (1,736 kB/s) Reading package lists... Reading package lists... Building dependency tree... Reading state information... The following additional packages will be installed: libtext-charwidth-perl Suggested packages: filters cowsay-off The following NEW packages will be installed: cowsay libtext-charwidth-perl sudo 0 upgraded, 3 newly installed, 0 to remove and 161 not upgraded. Need to get 1,275 kB of archives. After this operation, 4,018 kB of additional disk space will be used. Get:1 http://deb.debian.org/debian buster/main amd64 libtext-charwidth-perl amd64 0.04-7.1+b1 [10.0 kB] Get:2 http://deb.debian.org/debian buster/main amd64 cowsay all 3.03+dfsg2-6 [20.9 kB] Get:3 http://deb.debian.org/debian buster/main amd64 sudo amd64 1.8.27-1+deb10u3 [1,244 kB] debconf: delaying package configuration, since apt-utils is not installed Fetched 1,275 kB in 1s (913 kB/s) Selecting previously unselected package libtext-charwidth-perl. (Reading database ... 44180 files and directories currently installed.) Preparing to unpack .../libtext-charwidth-perl_0.04-7.1+b1_amd64.deb ... Unpacking libtext-charwidth-perl (0.04-7.1+b1) ... Selecting previously unselected package cowsay. Preparing to unpack .../cowsay_3.03+dfsg2-6_all.deb ... Unpacking cowsay (3.03+dfsg2-6) ... Selecting previously unselected package sudo. Preparing to unpack .../sudo_1.8.27-1+deb10u3_amd64.deb ... Unpacking sudo (1.8.27-1+deb10u3) ... Setting up libtext-charwidth-perl (0.04-7.1+b1) ... Setting up sudo (1.8.27-1+deb10u3) ... invoke-rc.d: could not determine current runlevel invoke-rc.d: policy-rc.d denied execution of start. Setting up cowsay (3.03+dfsg2-6) ... Processing triggers for man-db (2.8.5-2) ... Removing intermediate container 465908e3f8ee ---> a4efaec8307c Successfully built a4efaec8307c Successfully tagged extras:latest docker build --force-rm=true \ --build-arg=EXTRAS_IMG=extras \ --build-arg=UNAME=robbie.wu \ --build-arg=UID=701124824 \ --build-arg=GID=656741719 \ --build-arg=GROUP=656741719 \ -f dockerfiles/user.Dockerfile \ -t user_img-robbie.wu . Sending build context to Docker daemon 101.9kB Step 1/10 : ARG EXTRAS_IMG=extras Step 2/10 : FROM $EXTRAS_IMG ---> a4efaec8307c Step 3/10 : ARG UID ---> [Warning] The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested ---> Running in 3adf6b95270e Removing intermediate container 3adf6b95270e ---> d6096e545f04 Step 4/10 : ARG UNAME ---> [Warning] The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested ---> Running in 4e9b269f745c Removing intermediate container 4e9b269f745c ---> 2676b979f7c4 Step 5/10 : ARG GID ---> [Warning] The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested ---> Running in 08d11e3fed62 Removing intermediate container 08d11e3fed62 ---> 323b76ced3c5 Step 6/10 : ARG GROUP ---> [Warning] The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested ---> Running in 7b9e208c48cf Removing intermediate container 7b9e208c48cf ---> 50e8282edd6e Step 7/10 : COPY scripts/utils/user.sh /tmp/ ---> 9d8994b256a6 Step 8/10 : RUN /bin/bash /tmp/user.sh ---> [Warning] The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested ---> Running in ea8a1e164d5d

groupadd -fg 656741719 656741719
groupmod -g 656741719 656741719
IFS=:
read -r -a group_info ++ getent group 656741719
fgroup=656741719
fgid=656741719
GROUP_OK=false
'[' 656741719 == 656741719 ']'
'[' 656741719 == 656741719 ']'
GROUP_OK=true
'[' true = true ']'
useradd -u 701124824 -g 656741719 robbie.wu
passwd -d robbie.wu passwd: password expiry information changed.
cat
cat
mkdir /home/robbie.wu
cat
grep export /root/.bashrc
cat
chown_setting=robbie.wu
'[' true = true ']'
chown_setting=robbie.wu:656741719
mkdir -p /isabelle
chown -R robbie.wu:656741719 /isabelle
ln -s /isabelle /home/robbie.wu/.isabelle
chown -R robbie.wu:656741719 /home/robbie.wu
chmod -R ug+rw /home/robbie.wu

Removing intermediate container ea8a1e164d5d Error processing tar file(exit status 1): write /var/log/lastlog: no space left on device make: *** [build_user] Error 1 robbie.wu@MBFVFG30ELQ6LW seL4-CAmkES-L4v-dockerfiles % robbie.wu@MBFVFG30ELQ6LW seL4-CAmkES-L4v-dockerfiles %

robbiewu008 commented 3 years ago

docker run -ti --entrypoint bash trustworthysystems/sel4

It does NOT work for me.

robbie.wu@MBFVFG30ELQ6LW seL4-CAmkES-L4v-dockerfiles % docker run -ti --entrypoint bash trustworthysystems/sel4
Unable to find image 'trustworthysystems/sel4:latest' locally
latest: Pulling from trustworthysystems/sel4
afb6ec6fdc1c: Already exists
322dc1ce076f: Already exists
caf6234671a5: Already exists
fed4b0e4768f: Already exists
7a5b7604e264: Already exists
Digest: sha256:18d32c7154a55590625a36ce65221df46bad22b8426f899dcc72e683f46b1cf6
Status: Downloaded newer image for trustworthysystems/sel4:latest
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested

So this one looks successful to me, but the deciding bit would be the next line after this. For me it is:

root@...<image hex id>...:/#

i.e. if you see that, at that point you are inside the docker container.

I think the issue is the base image, which is for linux/amd64, could you try to provide the Dockerfile from the universal image?

OlympusMonds commented 3 years ago

Error processing tar file(exit status 1): write /var/log/lastlog: no space left on device

This implies your computer may have run out of disk space.

Try clearing out some of the images listed under docker image ls

lsf37 commented 3 years ago

I think the issue is the base image, which is for linux/amd64, could you try to provide the Dockerfile from the universal image?

We do not support building on ARM host machines, so I don't think we can build that image. The amd64 image does work for me on the M1, though.

Error processing tar file(exit status 1): write /var/log/lastlog: no space left on device This implies your computer may have run out of disk space.

Well spotted, I think this is correct. Looking at the rest of the trace, the build is proceeding as expected and got over the initial problem. I think we're close to getting it to work.

robbiewu008 commented 3 years ago

I think the issue is the base image, which is for linux/amd64, could you try to provide the Dockerfile from the universal image?

We do not support building on ARM host machines, so I don't think we can build that image. The amd64 image does work for me on the M1, though.

Yes, "./build.sh -b sel4 -r -v" does NOT work.

Error processing tar file(exit status 1): write /var/log/lastlog: no space left on device This implies your computer may have run out of disk space.

Well spotted, I think this is correct. Looking at the rest of the trace, the build is proceeding as expected and got over the initial problem. I think we're close to getting it to work.

Cannot get any clue.

robbie.wu@MBFVFG30ELQ6LW log % df -h Filesystem Size Used Avail Capacity iused ifree %iused Mounted on /dev/disk3s1s1 460Gi 14Gi 314Gi 5% 553781 4827422739 0% / devfs 203Ki 203Ki 0Bi 100% 706 0 100% /dev /dev/disk3s6 460Gi 1.0Gi 314Gi 1% 3 4827976517 0% /System/Volumes/VM /dev/disk3s2 460Gi 237Mi 314Gi 1% 707 4827975813 0% /System/Volumes/Preboot /dev/disk3s4 460Gi 2.2Mi 314Gi 1% 42 4827976478 0% /System/Volumes/Update /dev/disk1s2 500Mi 6.0Mi 481Mi 2% 4 5119996 0% /System/Volumes/xarts /dev/disk1s1 500Mi 7.4Mi 481Mi 2% 24 5119976 0% /System/Volumes/iSCPreboot /dev/disk1s3 500Mi 636Ki 481Mi 1% 42 5119958 0% /System/Volumes/Hardware /dev/disk3s5 460Gi 129Gi 314Gi 30% 659048 4827317472 0% /System/Volumes/Data map auto_home 0Bi 0Bi 0Bi 100% 0 0 100% /System/Volumes/Data/home map -fstab 0Bi 0Bi 0Bi 100% 0 0 100% /System/Volumes/Data/Network/Servers

lsf37 commented 3 years ago

/dev/disk3s5 460Gi 129Gi 314Gi 30% 659048 4827317472 0% /System/Volumes/Data

looks like there should be space available. Maybe it is the docker container volume that is out of space, not the host.

I'm assuming you're using Docker Desktop on the Mac. Could you open the preferences of Docker Desktop, and in the Resources panel increase the Disk image size to something very large (200GB) and then try make user again? (It shouldn't actually use that much, I just want to make sure it's definitely not that).

(I would leave CPU on 2 and memory should not need more than 8GB)

robbiewu008 commented 3 years ago

/dev/disk3s5 460Gi 129Gi 314Gi 30% 659048 4827317472 0% /System/Volumes/Data

looks like there should be space available. Maybe it is the docker container volume that is out of space, not the host.

I'm assuming you're using Docker Desktop on the Mac. Could you open the preferences of Docker Desktop, and in the Resources panel increase the Disk image size to something very large (200GB) and then try make user again? (It shouldn't actually use that much, I just want to make sure it's definitely not that).

(I would leave CPU on 2 and memory should not need more than 8GB)

Still container is out of space.

robbie.wu@MBFVFG30ELQ6LW seL4-CAmkES-L4v-dockerfiles % make user scripts/utils/check_for_old_docker_imgs.sh WARNING: Unable to check if your trustworthysystems docker images are getting a bit old! The date command did not behave as expected. Skipping the check. docker build --force-rm=true \ --build-arg=USER_BASE_IMG=trustworthysystems/camkes \ -f dockerfiles/extras.Dockerfile \ -t extras \ . Sending build context to Docker daemon 101.9kB Step 1/3 : ARG USER_BASE_IMG=trustworthysystems/sel4 Step 2/3 : FROM $USER_BASE_IMG ---> dd50eddb09be Step 3/3 : RUN apt-get update -q && apt-get install -y --no-install-recommends cowsay sudo ---> Using cache ---> 8d3dee8fd61b Successfully built 8d3dee8fd61b Successfully tagged extras:latest docker build --force-rm=true \ --build-arg=EXTRAS_IMG=extras \ --build-arg=UNAME=robbie.wu \ --build-arg=UID=701124824 \ --build-arg=GID=656741719 \ --build-arg=GROUP=656741719 \ -f dockerfiles/user.Dockerfile \ -t user_img-robbie.wu . Sending build context to Docker daemon 101.9kB Step 1/10 : ARG EXTRAS_IMG=extras Step 2/10 : FROM $EXTRAS_IMG ---> 8d3dee8fd61b Step 3/10 : ARG UID ---> Using cache ---> ccb34f074f63 Step 4/10 : ARG UNAME ---> Using cache ---> f5f48ba1d996 Step 5/10 : ARG GID ---> Using cache ---> 804f261498aa Step 6/10 : ARG GROUP ---> Using cache ---> 0ac9e95c474f Step 7/10 : COPY scripts/utils/user.sh /tmp/ ---> Using cache ---> 4dffa524a6a4 Step 8/10 : RUN /bin/bash /tmp/user.sh ---> [Warning] The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested ---> Running in c91d290f53bc

groupadd -fg 656741719 656741719
groupmod -g 656741719 656741719
IFS=:
read -r -a group_info ++ getent group 656741719
fgroup=656741719
fgid=656741719
GROUP_OK=false
'[' 656741719 == 656741719 ']'
'[' 656741719 == 656741719 ']'
GROUP_OK=true
'[' true = true ']'
useradd -u 701124824 -g 656741719 robbie.wu
passwd -d robbie.wu passwd: password expiry information changed.
cat
cat
mkdir /home/robbie.wu
cat
grep export /root/.bashrc
cat
chown_setting=robbie.wu
'[' true = true ']'
chown_setting=robbie.wu:656741719
mkdir -p /isabelle
chown -R robbie.wu:656741719 /isabelle
ln -s /isabelle /home/robbie.wu/.isabelle
chown -R robbie.wu:656741719 /home/robbie.wu
chmod -R ug+rw /home/robbie.wu Removing intermediate container c91d290f53bc Error processing tar file(exit status 1): write /var/log/lastlog: no space left on device make: *** [build_user] Error 1

lsf37 commented 3 years ago

Hm, getting towards the end of my docker knowledge. It's using cached images, so it might not pick up on the new size yet. https://stackoverflow.com/questions/30604846/docker-error-no-space-left-on-device suggests to run

docker system prune

robbiewu008 commented 3 years ago

Hm, getting towards the end of my docker knowledge. It's using cached images, so it might not pick up on the new size yet. https://stackoverflow.com/questions/30604846/docker-error-no-space-left-on-device suggests to run
docker system prune

still container is out space。

lsf37 commented 3 years ago

I'm out of ideas for now. Might try to precisely duplicate your settings later and see if that also runs out of space for me. @LukeMondy do you have an idea what could be going on?

lsf37 commented 3 years ago

Oh, actually: looking at your screenshot of the resources panel, it does say 198.1 GB of 200 GB used, so it is extremely close to full even with 200GB. Try increasing it all the way.

robbiewu008 commented 3 years ago

Oh, actually: looking at your screenshot of the resources panel, it does say 198.1 GB of 200 GB used, so it is extremely close to full even with 200GB. Try increasing it all the way.

when entering into the container, it hung at the "chmod -R ug+rw /home/robbie.wu" for more than 3 minutes, then got the out of space error, maybe the container was crashing repeatedly, tried to capture the crash dumps.

lsf37 commented 3 years ago

Still strange that it crashes, but I don't really have many more options to try. Does the smaller sel4 container work? I.e.

make user_sel4

robbiewu008 commented 3 years ago

Still strange that it crashes, but I don't really have many more options to try. Does the smaller sel4 container work? I.e.
make user_sel4

It does not work too.

Removing intermediate container e42beb29be90
Error processing tar file(exit status 1): write /var/log/lastlog: no space left on device
make: *** [build_user] Error 1

axel-h commented 2 years ago

Any update about this issue? Does the error happen only for @robbiewu008 or can somebody else reproduce this?

Jerryy959 commented 2 years ago

I have the same problem. The difference is that i use the debian through Parallels Desktop to simulate on my m1 pro. I get the warning "WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested"

lsf37 commented 2 years ago

I have the same problem. The difference is that i use the debian through Parallels Desktop to simulate on my m1 pro. I get the warning "WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested"

This sounds like a different problem. Are you saying you are also getting the same out-of-disk-space error afterwards?

chrisguikema commented 2 years ago

I was able to get applications building by compiling the Docker containers for an ARM host, but I had issues specifically with musllibc linking the syscall command, resulting in errors from the rootserver. I just had to add a few dependencies and compile the seL4 and camkes containers using the build.sh script, then add a default cross compiler toolchain for x86.

For the emulation route, using this yml setup I can enter the trustworthysystems/camkes container as a root user and compile seL4 images. Its slower than the native route, but functional. I just needed to increase the RAM to ~20GB to avoid the OOM killer stopping the make processes.

{
  "builder": {
    "gc": {
      "defaultKeepStorage": "20GB",
      "enabled": true
    }
  },
  "experimental": true,
  "features": {
    "buildkit": false
  }
}

chrisguikema commented 2 years ago

I was able to compile the containers for ARM. If there is sufficient interest I can push up a branch with some admittedly ugly changes.

lsf37 commented 2 years ago

I'd be interested to at least look at the necessary changes. Maybe we can make them less ugly. Do these changes break the x86 build or do we get images for both architectures?

chrisguikema commented 2 years ago

I am able to compile for x86 and ARM!

I opened an issue in musllibc here

For the seL4 kernel, I only needed to make this change (I know the code is a bit outdated, but it was the same when I pulled the camkes-manifest fresh)

diff --git a/gcc.cmake b/gcc.cmake
index 065f153d..24a11005 100644
--- a/gcc.cmake
+++ b/gcc.cmake
@@ -99,7 +99,10 @@ if("${CROSS_COMPILER_PREFIX}" STREQUAL "")
         if(APPLE)
             # APPLE is a CMake variable that evaluates to True on a Mac OSX system
             set(CROSS_COMPILER_PREFIX "x86_64-unknown-linux-gnu-")
-        endif()
+        elseif("${CMAKE_HOST_SYSTEM_PROCESSOR}" STREQUAL "aarch64")
+            # If we're on an aarch64 host, gcc won't support x86. Set an x86 compatible toolchain
+            set(CROSS_COMPILER_PREFIX "x86_64-linux-gnu-")
+       endif()

For whatever reason, the CMAKE_HOST_APPLE isn't set in the native aarch64 containers.

I forked the repo and pushed a branch here. Some of the changes are necessary, others aren't, especially if/when the fixes for musllibc and the kernel are integrated.

The biggest trick was updating the packages for an Arm host, but I found the right combination eventually. I'd love to get everything mainlined, as I think the M1 Mac is a popular software development platform

lsf37 commented 2 years ago

For whatever reason, the CMAKE_HOST_APPLE isn't set in the native aarch64 containers.

This bit I can explain: inside the container, the host is Linux, not Apple, so it wouldn't set that variable there. But the gcc.cmake patch looks like it should work for any aarch64 system (Linux or Apple), so that part is good, I think.

chrisguikema commented 2 years ago

That makes sense. I was a bit confused because CMAKE_HOST_APPLE was set when emulating the x86 containers.

Please let me know if you need any help or if you want me to make a PR on musllibc

chrisguikema commented 1 year ago

@lsf37 Any update? I noticed there were a lot of MacBook users at the Summit, so just wondering if the Docker container will officially support M1 Macs anytime soon

lsf37 commented 1 year ago

@Ivan-Velickovic was working on M1 support for the containers and got most of it working apart from Haskell stack. @Ivan-Velickovic any update on that?

chrisguikema commented 1 year ago

Yeah, I had to pull down a tarball since the Haskell install script didn't support ARM hosts.

Ivan-Velickovic commented 1 year ago

@Ivan-Velickovic was working on M1 support for the containers and got most of it working apart from Haskell stack. @Ivan-Velickovic any update on that?

No, I couldn't figure out. It seems like other people also have this issue (e.g https://github.com/commercialhaskell/stack/issues/5734) but there seems to be no resolution. It's probably better to go with @chrisguikema changes to get an ARM container working rather than adding another layer of software to do x86 to ARM translation.

chrisguikema commented 1 year ago

I also made a PR in the seL4 kernel repo to address to x86 cross-compiler issue.

@Ivan-Velickovic are you seeing the issue with sed in musllibc?

sed: couldn't open temporary file ./sedVTt6YE: Permission denied

Ivan-Velickovic commented 1 year ago

I also made a PR in the seL4 kernel repo to address to x86 cross-compiler issue.

@Ivan-Velickovic are you seeing the issue with sed in musllibc?
sed: couldn't open temporary file ./sedVTt6YE: Permission denied

When I use your patch to get an ARM image (https://github.com/chrisguikema/seL4-CAmkES-L4v-dockerfiles/commit/144bfe1a836e7e5ae57ac8a63cb2816c6f3578cd), I still get an x86_64 Linux container. Not sure what's going on.

chrisguikema commented 1 year ago

Just to clarify, you aren't pulling down prebuilts from Dockerhub, right? Specifying the --platform flag should force an Arm container

tiferrei commented 1 year ago

Hi, are there any news regarding the best practises in building seL4 on Apple Silicon Macs?

I think we'd all hope to be able to do fully native > docker arm image > docker x86 image (performance wise on the mac), however right now I'm struggling with even running the sel4test example on docker x86 images. They build (with some tweaks to the Makefile), but running ./simulate crashes soon after the ROM boot.

Oddly, compiling for an arm target in the x86 docker container seems to run the tests fine. I gave the modified scripts (https://github.com/chrisguikema/seL4-CAmkES-L4v-dockerfiles/commit/144bfe1a836e7e5ae57ac8a63cb2816c6f3578cd) a try, but they don't seem to build for me, as one of the images can't find the arm compilers on apt-get.

If I then run ./simulate of an arm build from an x86 image, locally on the Mac, it runs all tests perfectly at lightning speed.

I've also noticed that things seem to compile fine (natively on M1) by just adapting the date parameter of touch to -d 1970-01-01T00:00:00, but I image other seL4 things beyond seL4test might need further adjusting.

chrisguikema commented 1 year ago

What is the specific error you're seeing with compiling the Docker containers for ARM?

tiferrei commented 1 year ago

Sorry @chrisguikema, I was in the wrong branch! All compiling fine now. Although I do see the same thing as @Ivan-Velickovic, although docker metadata reports an arm image, all the binaries inside are x86 and uname -a still reports x86. Which I found quite weird. Nonetheless, I will call these "ARM" images in this context.

Out of curiosity I ran this (very anecdotal, not at all scientific) benchmark: Compile and simulate sel4test for ARM with different platforms. The results are as expected regarding native vs emulated. Not so expected on the differences by docker backed by QEMU vs backed by Apple virtualisation framework:

Platform	Emulator	Architecture	Compile Time	Simulation Time
macOS	Native	ARM	`0m3.1s`	`0m36.6s`
Docker	QEMU	ARM	`0m31.0s`	`0m39.9s`
Docker	QEMU	x86	`6m38.5s`	`7m21.0s`
Docker	Apple Virt.	ARM	`0m17.8s`	`0m38.7s`
Docker	Apple Virt.	x86	`0m23.5s`	`6m48.3s`

I'd be curious to see if maybe this is related the ARM images still having x86 binaries.

chrisguikema commented 1 year ago

Interesting. Here is my uname -a from the container.

chrisguikema@in-container: /host  $ uname -a
Linux in-container 5.10.104-linuxkit #1 SMP PREEMPT Thu Mar 17 17:05:54 UTC 2022 aarch64 GNU/Linux

tiferrei commented 1 year ago

Ah, I built it again from a fresh docker installation and it probably cleared whatever cache was pulling an x86 base image.

I have updated the table with the new figures (much better!)

The new results show a massive improvement in using the ARM images. Simulating time is almost on part with native, and compile time is only about 5x slower (when using Apple virt), as opposed to 7x.

lsf37 commented 4 months ago

65 is adding ARM64 variants of all of the base images we have. It's not a full multi-arch image yet that you can pull from docker hub, but you should now be able to build native containers for Apple Silicon as well as the the normal Intel/AMD containers.

Multi-arch images will be the next step once this works properly in CI.

seL4 / seL4-CAmkES-L4v-dockerfiles

make user failed on MacBook Pro with Apple M1 chipset #40

It's OK with my docker environment.

It's also OK to build a getting-started images:

It does NOT work for me.

it does NOT work for me too.

[internal] load metadata for docker.io/library/extras:latest:

It does NOT work for me.

still does not work.

It does NOT work for me.

65 is adding ARM64 variants of all of the base images we have. It's not a full multi-arch image yet that you can pull from docker hub, but you should now be able to build native containers for Apple Silicon as well as the the normal Intel/AMD containers.