Open vicshi06 opened 4 months ago
I do not see anything special about your original Docker image.
Would it be possible to reproduce the same on a clean environment (with all previously built Docker images purged)? And attach the GSC-build command that you used as well as the full GSC build log.
Yes. I tried to remove all of my images and built it, but it gave the same error.
Here is the only command I used:
gsc build --insecure-args vicshi06/test:debian ../graminized/plugin.manifest
Here is the full log:
Building unsigned graminized Docker image `gsc-vicshi06/test:debian-unsigned` from original application image `vicshi06/test:debian`...
Warning: Duplicate key `sgx.debug`. Overriding value from `debian/entrypoint.manifest.template` by the one in `../graminized/plugin.manifest`.
Warning: Duplicate key `loader.env.PATH`. Concatenating values from `<merged ../graminized/plugin.manifest and debian/entrypoint.manifest.template>` and `<vicshi06/test:debian image env>`.
Step 1/29 : FROM debian:12 AS gramine
---> e15dbfac2d2b
Step 2/29 : RUN env DEBIAN_FRONTEND=noninteractive apt-get update && env DEBIAN_FRONTEND=noninteractive apt-get install -y autoconf bison build-essential coreutils curl gawk git libprotobuf-c-dev linux-headers-generic nasm ninja-build pkg-config protobuf-c-compiler protobuf-compiler python3 python3-cryptography python3-protobuf wget meson python3-tomli python3-tomli-w
---> Using cache
---> 0793bd44f91c
Step 3/29 : COPY intel-sgx-deb.key /
---> Using cache
---> 0021a175fff3
Step 4/29 : RUN echo 'deb [arch=amd64] https://download.01.org/intel-sgx/sgx_repo/ubuntu bionic main' > /etc/apt/sources.list.d/intel-sgx.list && apt-key add /intel-sgx-deb.key
---> Using cache
---> 14e5b9d262cc
Step 5/29 : RUN env DEBIAN_FRONTEND=noninteractive apt-get update && env DEBIAN_FRONTEND=noninteractive apt-get install -y libsgx-dcap-quote-verify-dev
---> Using cache
---> 2619c6c81725
Step 6/29 : RUN git clone https://github.com/gramineproject/gramine.git /gramine
---> Using cache
---> faf3c4726b78
Step 7/29 : RUN cd /gramine && git fetch origin master && git checkout master
---> Using cache
---> f47f34852c74
Step 8/29 : RUN mkdir -p /gramine/driver/asm && cd /gramine/driver/asm && wget --timeout=10 -O sgx.h https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/plain/arch/x86/include/uapi/asm/sgx.h?h=v5.11 && sha256sum sgx.h | grep -q a34a997ade42b61376b1c5d3d50f839fd28f2253fa047cb9c0e68a1b00477956
---> Using cache
---> daaed81bb6fe
Step 9/29 : RUN cd /gramine && meson setup build/ --prefix="/gramine/meson_build_output" --buildtype=release -Ddirect=enabled -Dsgx=enabled -Dsgx_driver=upstream -Dsgx_driver_include_path=/gramine/driver && ninja -C build && ninja -C build install
---> Using cache
---> bba0e9d779fc
Step 10/29 : FROM vicshi06/test:debian
---> 5a11f11890b8
Step 11/29 : USER root
---> Using cache
---> b2e95142dae6
Step 12/29 : RUN apt-get update && env DEBIAN_FRONTEND=noninteractive apt-get install -y binutils libprotobuf-c-dev locales openssl python3 python3-cryptography python3-protobuf python3-pyelftools python3-click python3-jinja2 python3-tomli python3-tomli-w && apt-get autoremove -y && rm -rf /var/lib/apt/lists/*
---> Running in 091f3aaa244e
Get:1 http://deb.debian.org/debian bookworm InRelease [151 kB]
Err:1 http://deb.debian.org/debian bookworm InRelease
Couldn't create temporary file /tmp/apt.conf.CfqgCs for passing config to apt-key
Get:2 http://deb.debian.org/debian bookworm-updates InRelease [55.4 kB]
Err:2 http://deb.debian.org/debian bookworm-updates InRelease
Couldn't create temporary file /tmp/apt.conf.nNjBHj for passing config to apt-key
Get:3 http://deb.debian.org/debian-security bookworm-security InRelease [48.0 kB]
Err:3 http://deb.debian.org/debian-security bookworm-security InRelease
Couldn't create temporary file /tmp/apt.conf.0mPqVF for passing config to apt-key
Reading package lists...
W: GPG error: http://deb.debian.org/debian bookworm InRelease: Couldn't create temporary file /tmp/apt.conf.CfqgCs for passing config to apt-key
E: The repository 'http://deb.debian.org/debian bookworm InRelease' is not signed.
W: GPG error: http://deb.debian.org/debian bookworm-updates InRelease: Couldn't create temporary file /tmp/apt.conf.nNjBHj for passing config to apt-key
E: The repository 'http://deb.debian.org/debian bookworm-updates InRelease' is not signed.
W: GPG error: http://deb.debian.org/debian-security bookworm-security InRelease: Couldn't create temporary file /tmp/apt.conf.0mPqVF for passing config to apt-key
E: The repository 'http://deb.debian.org/debian-security bookworm-security InRelease' is not signed.
Failed to build unsigned graminized Docker image `gsc-vicshi06/test:debian-unsigned`.
Here is my plugin.manifest:
[loader]
log_level = "error"
[fs]
mounts = [
{ type = "tmpfs", path = "/tmp/" },
]
[sgx]
debug = true
edmm_enable = false
enclave_size = "4G"
max_threads = 512
remote_attestation = "dcap"
allowed_files = [
"file:/etc/nsswitch.conf",
"file:/etc/host.conf",
"file:/etc/resolv.conf",
"file:/etc/ethers",
"file:/etc/hosts",
"file:/etc/group",
"file:/etc/passwd",
"file:/etc/gai.conf",
"file:/etc/ssl/certs/ca-certificates.crt",
]
trusted_files = []
isvprodid = 0
isvsvn = 0
enable_stats = false
use_exinfo = false
[sgx.cpu_features]
avx = "unspecified"
avx512 = "unspecified"
amx = "unspecified"
mpx = "disabled"
pkru = "disabled"
RUN chmod 1777 /tmp
Do you know why you're doing this in your Docker image build?
@vicshi06 Can you check if PR https://github.com/gramineproject/gsc/pull/194 fixes your issue?
@vicshi06 Can you check, why the image has wrong mode on /tmp
? Because debian:12
looks good to me:
% docker run -it --rm debian:12 ls -ld /tmp
drwxrwxrwt 2 root root 4096 Jun 12 2023 /tmp
So I assume the problem is somewhere between FROM
and the resulting image.
RUN chmod 1777 /tmp
Do you know why you're doing this in your Docker image build?
@dimakuv I added this after I encountered the bug, but it didn't solve the issue.
@vicshi06 Can you check, why the image has wrong mode on
/tmp
? Becausedebian:12
looks good to me:% docker run -it --rm debian:12 ls -ld /tmp drwxrwxrwt 2 root root 4096 Jun 12 2023 /tmp
So I assume the problem is somewhere between
FROM
and the resulting image.
@woju As you can see, I didn't even touch /tmp
in my final image. All I am doing is copying files from builder to my image. I don't understand why this would change mode on /tmp
. In addition, I am using the official Dockerfile from the spire repo, but the only two things I changed are using debian:12
and mounting my own binary.
This is quite a weird bug :(
@vicshi06 Can you check if PR #194 fixes your issue?
@dimakuv It did fix the issue, but now the build failed at step 23 with no helpful error message
Building unsigned graminized Docker image `gsc-vicshi06/test:debian-unsigned` from original application image `vicshi06/test:debian`...
Warning: Duplicate key `sgx.debug`. Overriding value from `debian/entrypoint.manifest.template` by the one in `../graminized/plugin.manifest`.
Warning: Duplicate key `loader.env.PATH`. Concatenating values from `<merged ../graminized/plugin.manifest and debian/entrypoint.manifest.template>` and `<vicshi06/test:debian image env>`.
Step 1/30 : FROM debian:12 AS gramine
---> e15dbfac2d2b
Step 2/30 : RUN env DEBIAN_FRONTEND=noninteractive apt-get update && env DEBIAN_FRONTEND=noninteractive apt-get install -y autoconf bison build-essential coreutils curl gawk git libprotobuf-c-dev linux-headers-generic nasm ninja-build pkg-config protobuf-c-compiler protobuf-compiler python3 python3-cryptography python3-protobuf wget meson python3-tomli python3-tomli-w
---> Using cache
---> 0793bd44f91c
Step 3/30 : COPY intel-sgx-deb.key /
---> Using cache
---> 0021a175fff3
Step 4/30 : RUN echo 'deb [arch=amd64] https://download.01.org/intel-sgx/sgx_repo/ubuntu bionic main' > /etc/apt/sources.list.d/intel-sgx.list && apt-key add /intel-sgx-deb.key
---> Using cache
---> 14e5b9d262cc
Step 5/30 : RUN env DEBIAN_FRONTEND=noninteractive apt-get update && env DEBIAN_FRONTEND=noninteractive apt-get install -y libsgx-dcap-quote-verify-dev
---> Using cache
---> 2619c6c81725
Step 6/30 : RUN git clone https://github.com/gramineproject/gramine.git /gramine
---> Using cache
---> faf3c4726b78
Step 7/30 : RUN cd /gramine && git fetch origin master && git checkout master
---> Using cache
---> f47f34852c74
Step 8/30 : RUN mkdir -p /gramine/driver/asm && cd /gramine/driver/asm && wget --timeout=10 -O sgx.h https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/plain/arch/x86/include/uapi/asm/sgx.h?h=v5.11 && sha256sum sgx.h | grep -q a34a997ade42b61376b1c5d3d50f839fd28f2253fa047cb9c0e68a1b00477956
---> Using cache
---> daaed81bb6fe
Step 9/30 : RUN cd /gramine && meson setup build/ --prefix="/gramine/meson_build_output" --buildtype=release -Ddirect=enabled -Dsgx=enabled -Dsgx_driver=upstream -Dsgx_driver_include_path=/gramine/driver && ninja -C build && ninja -C build install
---> Using cache
---> bba0e9d779fc
Step 10/30 : FROM vicshi06/test:debian
---> 5a11f11890b8
Step 11/30 : USER root
---> Using cache
---> b2e95142dae6
Step 12/30 : RUN chmod 1777 /tmp
---> Using cache
---> 06ab4bdd8edc
Step 13/30 : RUN apt-get update && env DEBIAN_FRONTEND=noninteractive apt-get install -y binutils libprotobuf-c-dev locales openssl python3 python3-cryptography python3-protobuf python3-pyelftools python3-click python3-jinja2 python3-tomli python3-tomli-w && apt-get autoremove -y && rm -rf /var/lib/apt/lists/*
---> Using cache
---> 314c90963475
Step 14/30 : RUN locale-gen en_US.UTF-8
---> Using cache
---> 0b2a3dea6325
Step 15/30 : ENV LC_ALL en_US.UTF-8
---> Using cache
---> c22e753beada
Step 16/30 : ENV LANG en_US.UTF-8
---> Using cache
---> f419e73ab5ef
Step 17/30 : ENV LANGUAGE en_US.UTF-8
---> Using cache
---> a67c71d131fa
Step 18/30 : RUN mkdir -p /gramine/app_files
---> Using cache
---> a9a50ef74ad0
Step 19/30 : RUN chown : /gramine/app_files/
---> Using cache
---> 77f589fa48e9
Step 20/30 : RUN rm -rf $HOME/.cache
---> Using cache
---> 86e07c52514f
Step 21/30 : USER :
---> Using cache
---> 9954f69dd2a4
Step 22/30 : RUN rm -rf $HOME/.cache
---> Using cache
---> 2875cb2ec6df
Step 23/30 : COPY --from=gramine --chown=: /gramine/meson_build_output /gramine/meson_build_output
Failed to build unsigned graminized Docker image `gsc-vicshi06/test:debian-unsigned`.
It feels like the USER in the original Docker image is empty or corrupted? Why do we see lines like Step 21/30 : USER :
...
Can you perform docker inspect vicshi06/test:debian
on your original Docker image and show us the output?
docker inspect vicshi06/test:debian
You are right. For some reason my user is ":"
[
{
"Id": "sha256:5a11f11890b8dc47e53a01408bd65573206cef787b94d280c1dcf396f56724ef",
"RepoTags": [
"vicshi06/test:debian"
],
"RepoDigests": [
"vicshi06/test@sha256:b75d1efc18fb1f43f145cd12a404c7345165f22b90018a8954628df2d3fc7d46"
],
"Parent": "",
"Comment": "buildkit.dockerfile.v0",
"Created": "2024-04-12T23:09:54.203402651Z",
"Container": "",
"ContainerConfig": {
"Hostname": "",
"Domainname": "",
"User": "",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"Tty": false,
"OpenStdin": false,
"StdinOnce": false,
"Env": null,
"Cmd": null,
"Image": "",
"Volumes": null,
"WorkingDir": "",
"Entrypoint": null,
"OnBuild": null,
"Labels": null
},
"DockerVersion": "",
"Author": "",
"Config": {
"Hostname": "",
"Domainname": "",
"User": ":",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"Tty": false,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
],
"Cmd": null,
"ArgsEscaped": true,
"Image": "",
"Volumes": null,
"WorkingDir": "/opt/spire",
"Entrypoint": [
"/opt/spire/bin/spire-agent",
"run"
],
"OnBuild": null,
"Labels": null
},
"Architecture": "amd64",
"Os": "linux",
"Size": 194414300,
"VirtualSize": 194414300,
"GraphDriver": {
"Data": {
"LowerDir": "/var/lib/docker/overlay2/96095bc8f734226c75c4ce40508134ca8dd855c3b118a6079f4547baa3588892/diff:/var/lib/docker/overlay2/a2a52730ddb817dbf32bf19865fe003c1b2e8d0e976fc37b91596e88bba39538/diff:/var/lib/docker/overlay2/7aa7c5846dd34bec8298258288e63cdb84637fa0883717f6826f8b046cdb3801/diff:/var/lib/docker/overlay2/31f0c2685a7634560f5d11b3ceda28f6b0264deed38d3d2709cd77e937cba53b/diff:/var/lib/docker/overlay2/4ce3380a8a629623e39710115110cbe943eae63a4d424cb94347eb0e7f31a1d7/diff:/var/lib/docker/overlay2/b1f0a51b85e8b1b378bed5f7bcf1eb4aec4bd3dcf8ffcf7b1fa051168b3bee59/diff",
"MergedDir": "/var/lib/docker/overlay2/cf0a1415cf5d3f874a211e91316edee9df9ece595d5357248da8404e54e15854/merged",
"UpperDir": "/var/lib/docker/overlay2/cf0a1415cf5d3f874a211e91316edee9df9ece595d5357248da8404e54e15854/diff",
"WorkDir": "/var/lib/docker/overlay2/cf0a1415cf5d3f874a211e91316edee9df9ece595d5357248da8404e54e15854/work"
},
"Name": "overlay2"
},
"RootFS": {
"Type": "layers",
"Layers": [
"sha256:c5bb35826823702969891b025087135cceefb084dec1452af5a6fb2938bd9a13",
"sha256:8ebdd97378f6249ddcd979555a1554cc328e3cab204dc3240526d25e96897fa2",
"sha256:9f7679fd96b1d89231c89068e7b8b4fa6d7abfb021edeca4aae698acfd54d9ef",
"sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef",
"sha256:aec261fe06308117d48efce9f215bd0e2f6505fe8eb48f8e6fc28b9339c9e22e",
"sha256:0805e59bcae86efa5f513075a300af9ade6b02b35fee97788cbb1e3b1af92a05",
"sha256:b317d793126fad6559aeb3d5c6856cca49ca2e5bf0dc88fd9e30accbb9d20163"
]
},
"Metadata": {
"LastTagTime": "0001-01-01T00:00:00Z"
}
}
]
Description of the problem
We have been trying to use gramine and gsc to build our custom graminized image. However, when we use Debian:12 as the base image, it gives me this error. It is the similar error if we use Ubuntu. The build process failed at step 12.
The image works fine if I directly use Docker run on it.
Steps to reproduce
Here is our Dockerfile. We use Debian:12 right before we build the SPIRE-agent.
This is my config.yaml for GSC: