cross-rs / cross-toolchains

Additional Dockerfiles and crosstool-ng config files to build additional toolchains.
Apache License 2.0
88 stars 16 forks source link

x86_64-apple-darwin-cross manifest unknown #48

Closed frenchtoastbeer closed 8 months ago

frenchtoastbeer commented 8 months ago

Checklist

Describe your issue

I'm seeing this when I try to build the darwin image:

$ cargo build-docker-image x86_64-apple-darwin-cross --build-arg='MACOS_SDK_DIR=MacOSX-SDKs' --build-arg 'MACOS_SDK_FILE=MacOSX11.3.sdk.tar.xz' --tag local
    Finished dev [unoptimized + debuginfo] target(s) in 0.02s
     Running `target/debug/xtask build-docker-image x86_64-apple-darwin-cross --build-arg=MACOS_SDK_DIR=MacOSX-SDKs --build-arg MACOS_SDK_FILE=MacOSX11.3.sdk.tar.xz --tag local`
[cross] note: Build x86_64-apple-darwin-cross for x86_64-unknown-linux-gnu
[+] Building 2.9s (14/19)                                                                                                                                                                                                                           docker-container:xenodochial_wilson
 => [internal] load build definition from Dockerfile.x86_64-apple-darwin-cross                                                                                                                                                                                                     0.0s
 => => transferring dockerfile: 1.17kB                                                                                                                                                                                                                                             0.0s
 => [internal] load metadata for docker.io/library/ubuntu:20.04                                                                                                                                                                                                                    0.3s
 => [internal] load .dockerignore                                                                                                                                                                                                                                                  0.0s
 => => transferring context: 223B                                                                                                                                                                                                                                                  0.0s
 => ERROR importing cache manifest from ghcr.io/cross-rs/x86_64-apple-darwin-cross:main                                                                                                                                                                                            0.6s
 => [internal] load build context                                                                                                                                                                                                                                                  1.8s
 => => transferring context: 365B                                                                                                                                                                                                                                                  1.8s
 => [ 1/13] FROM docker.io/library/ubuntu:20.04@sha256:f2034e7195f61334e6caff6ecf2e965f92d11e888309065da85ff50c617732b8                                                                                                                                                            0.0s
 => => resolve docker.io/library/ubuntu:20.04@sha256:f2034e7195f61334e6caff6ecf2e965f92d11e888309065da85ff50c617732b8                                                                                                                                                              0.0s
 => [auth] cross-rs/x86_64-apple-darwin-cross:pull token for ghcr.io                                                                                                                                                                                                               0.0s
 => CACHED [ 2/13] COPY common.sh lib.sh /                                                                                                                                                                                                                                         0.0s
 => CACHED [ 3/13] RUN /common.sh                                                                                                                                                                                                                                                  0.0s
 => CACHED [ 4/13] COPY cmake.sh /                                                                                                                                                                                                                                                 0.0s
 => CACHED [ 5/13] RUN /cmake.sh                                                                                                                                                                                                                                                   0.0s
 => CACHED [ 6/13] COPY xargo.sh /                                                                                                                                                                                                                                                 0.0s
 => CACHED [ 7/13] RUN /xargo.sh                                                                                                                                                                                                                                                   0.0s
 => ERROR [ 8/13] COPY MacOSX-SDKs/MacOSX11.3.sdk.tar.xz* /                                                                                                                                                                                                                        0.0s
------
 > importing cache manifest from ghcr.io/cross-rs/x86_64-apple-darwin-cross:main:
------
------
 > [ 8/13] COPY MacOSX-SDKs/MacOSX11.3.sdk.tar.xz* /:
------
Dockerfile.x86_64-apple-darwin-cross:21
--------------------
  19 |     ARG MACOS_SDK_URL
  20 |     # wildcard workaround so we can copy the file only if it exists
  21 | >>> COPY $MACOS_SDK_DIR/$MACOS_SDK_FILE* /
  22 |     COPY cross-toolchains/docker/darwin.sh /
  23 |     RUN /darwin.sh
--------------------
ERROR: failed to solve: lstat /tmp/buildkit-mount2628248203/MacOSX-SDKs: no such file or directory
Error: 
   0: `docker buildx build --platform linux/amd64 --build-arg 'CROSS_TARGET_TRIPLE=X86_64_APPLE_DARWIN_CROSS' --load --pull --cache-from 'type=registry,ref=ghcr.io/cross-rs/x86_64-apple-darwin-cross:main' --tag ghcr.io/cross-rs/x86_64-apple-darwin-cross:local --label 'org.cross-rs.for-cross-target=x86_64-apple-darwin-cross' --label 'org.cross-rs.runs-with=x86_64-unknown-linux-gnu' --file /home/fr3ncht0ast/cross/docker/cross-toolchains/docker/Dockerfile.x86_64-apple-darwin-cross --progress auto --build-arg 'MACOS_SDK_DIR=MacOSX-SDKs' --build-arg 'MACOS_SDK_FILE=MacOSX11.3.sdk.tar.xz' .` failed with exit status: 1

Location:
   xtask/src/build_docker_image.rs:309

Warning: call to docker failed
Suggestion: is `buildx` available for the container engine?
Note: disable the `buildkit` dependency optionally with `CROSS_CONTAINER_ENGINE_NO_BUILDKIT=1`

Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it.
Run with RUST_BACKTRACE=full to include source snippets.

I was surprised to see ERROR importing cache manifest from ghcr.io/cross-rs/x86_64-apple-darwin-cross:main and then attempted to check the manifest to see what the issue was:

$ docker manifest inspect ghcr.io/cross-rs/x86_64-apple-darwin-cross
manifest unknown

I believe I will be unable to build the docker image with the requisite SDKs and am unsure how I could proceed.

What target(s) are you cross-compiling for?

other (specify in description)

Which operating system is the host (e.g computer cross is on) running?

What architecture is the host?

What container engine is cross using?

cross version

cross 0.2.5 (ade8dbe 2024-01-02)

Example

$ docker login
Authenticating with existing credentials...
Login Succeeded
$ echo "###" | docker login ghcr.io -u "####" --password-stdin
Login Succeeded
$ git clone https://github.com/cross-rs/cross
$ cd cross
$ git submodule update --init --remote
  grabbed the SDK dir and file, downloaded to: 
   ./docker/cross-toolchains/docker/MacOSX-SDKs and 
   ./docker/cross-toolchains/docker/MacOSX11.3.sdk.tar.xz
$ MACOS_SDK_DIR=MacOSX-SDKs
$ MACOS_SDK_FILE=MacOSX11.3.sdk.tar.xz
$ cargo build-docker-image x86_64-apple-darwin-cross \
   --build-arg 'MACOS_SDK_DIR=$DIR' \
   --build-arg 'MACOS_SDK_FILE=$FILE'  \
   --tag local

Additional information / notes

I'm not 100% certain that this isn't still my fault somehow.

frenchtoastbeer commented 8 months ago

perhaps also relevant, here's what happens when I run the docker buildx build command directly:

$ cd docker/cross-toolchains/docker/
$ docker buildx build --platform linux/amd64 --build-arg 'CROSS_TARGET_TRIPLE=X86_64_APPLE_DARWIN_CROSS' --load --pull --cache-from 'type=registry,ref=ghcr.io/cross-rs/x86_64-apple-darwin-cross:main' --tag ghcr.io/cross-rs/x86_64-apple-darwin-cross:local --label 'org.cross-rs.for-cross-target=x86_64-apple-darwin-cross' --label 'org.cross-rs.runs-with=x86_64-unknown-linux-gnu' --file /home/fr3ncht0ast/cross/docker/cross-toolchains/docker/Dockerfile.x86_64-apple-darwin-cross --progress auto --build-arg 'MACOS_SDK_DIR=MacOSX-SDKs' --build-arg 'MACOS_SDK_FILE=MacOSX11.3.sdk.tar.xz' .
[+] Building 4.1s (20/20) FINISHED                                                                                                                                                                                                                  docker-container:xenodochial_wilson
 => [internal] load build definition from Dockerfile.x86_64-apple-darwin-cross                                                                                                                                                                                                     0.0s
 => => transferring dockerfile: 1.17kB                                                                                                                                                                                                                                             0.0s
 => [internal] load metadata for docker.io/library/ubuntu:20.04                                                                                                                                                                                                                    0.7s
 => [auth] library/ubuntu:pull token for registry-1.docker.io                                                                                                                                                                                                                      0.0s
 => [internal] load .dockerignore                                                                                                                                                                                                                                                  0.0s
 => => transferring context: 2B                                                                                                                                                                                                                                                    0.0s
 => ERROR importing cache manifest from ghcr.io/cross-rs/x86_64-apple-darwin-cross:main                                                                                                                                                                                            2.8s
 => [internal] load build context                                                                                                                                                                                                                                                  0.6s
 => => transferring context: 2B                                                                                                                                                                                                                                                    0.6s
 => [ 1/13] FROM docker.io/library/ubuntu:20.04@sha256:f2034e7195f61334e6caff6ecf2e965f92d11e888309065da85ff50c617732b8                                                                                                                                                            0.0s
 => => resolve docker.io/library/ubuntu:20.04@sha256:f2034e7195f61334e6caff6ecf2e965f92d11e888309065da85ff50c617732b8                                                                                                                                                              0.0s
 => [auth] cross-rs/x86_64-apple-darwin-cross:pull token for ghcr.io                                                                                                                                                                                                               0.0s
 => ERROR [ 2/13] COPY common.sh lib.sh /                                                                                                                                                                                                                                          0.0s
 => CACHED [ 3/13] RUN /common.sh                                                                                                                                                                                                                                                  0.0s
 => ERROR [ 4/13] COPY cmake.sh /                                                                                                                                                                                                                                                  0.0s
 => CACHED [ 5/13] RUN /cmake.sh                                                                                                                                                                                                                                                   0.0s
 => ERROR [ 6/13] COPY xargo.sh /                                                                                                                                                                                                                                                  0.0s
 => CACHED [ 7/13] RUN /xargo.sh                                                                                                                                                                                                                                                   0.0s
 => CACHED [ 8/13] COPY MacOSX-SDKs/MacOSX11.3.sdk.tar.xz* /                                                                                                                                                                                                                       0.0s
 => ERROR [ 9/13] COPY cross-toolchains/docker/darwin.sh /                                                                                                                                                                                                                         0.0s
 => CACHED [10/13] RUN /darwin.sh                                                                                                                                                                                                                                                  0.0s
 => ERROR [11/13] COPY cross-toolchains/docker/darwin-symlink.sh /                                                                                                                                                                                                                 0.0s
 => CACHED [12/13] RUN /darwin-symlink.sh                                                                                                                                                                                                                                          0.0s
 => ERROR [13/13] COPY cross-toolchains/docker/darwin-entry.sh /                                                                                                                                                                                                                   0.0s
------
 > importing cache manifest from ghcr.io/cross-rs/x86_64-apple-darwin-cross:main:
------
------
 > [ 2/13] COPY common.sh lib.sh /:
------
------
 > [ 4/13] COPY cmake.sh /:
------
------
 > [ 6/13] COPY xargo.sh /:
------
------
 > [ 9/13] COPY cross-toolchains/docker/darwin.sh /:
------
------
 > [11/13] COPY cross-toolchains/docker/darwin-symlink.sh /:
------
------
 > [13/13] COPY cross-toolchains/docker/darwin-entry.sh /:
------
Dockerfile.x86_64-apple-darwin-cross:32
--------------------
  30 |         CROSS_TARGET=x86_64-apple-darwin
  31 |     
  32 | >>> COPY cross-toolchains/docker/darwin-entry.sh /
  33 |     ENTRYPOINT ["/darwin-entry.sh"]
  34 |     
--------------------
ERROR: failed to solve: failed to compute cache key: failed to calculate checksum of ref i8fbbomet2zr3q5x29vgg5vt9::lraikh56rgj60r1h8toc88ax4: "/cross-toolchains/docker/darwin-entry.sh": not found
Emilgardis commented 8 months ago

Moved this to cross-toolchains.

The issue you're seeing with ERROR importing cache manifest from ghcr.io/cross-rs/x86_64-apple-darwin-cross:main is not an error and doesn't cause any problems, it simply means that there's no cache available on ghcr.io/cross-rs/x86_64-apple-darwin-cross:main (rightfully so also)

The other issue when you're calling docker build directly is an issue with how you're invoking build.

The problem you're actually experiencing is

Dockerfile.x86_64-apple-darwin-cross:21
--------------------
  19 |     ARG MACOS_SDK_URL
  20 |     # wildcard workaround so we can copy the file only if it exists
  21 | >>> COPY $MACOS_SDK_DIR/$MACOS_SDK_FILE* /
  22 |     COPY cross-toolchains/docker/darwin.sh /
  23 |     RUN /darwin.sh
--------------------
ERROR: failed to solve: lstat /tmp/buildkit-mount2628248203/MacOSX-SDKs: no such file or directory
Error: 
   0: `docker buildx build --platform linux/amd64 --build-arg 'CROSS_TARGET_TRIPLE=X86_64_APPLE_DARWIN_CROSS' --load --pull --cache-from 'type=registry,ref=ghcr.io/cross-rs/x86_64-apple-darwin-cross:main' --tag ghcr.io/cross-rs/x86_64-apple-darwin-cross:local --label 'org.cross-rs.for-cross-target=x86_64-apple-darwin-cross' --label 'org.cross-rs.runs-with=x86_64-unknown-linux-gnu' --file /home/fr3ncht0ast/cross/docker/cross-toolchains/docker/Dockerfile.x86_64-apple-darwin-cross --progress auto --build-arg 'MACOS_SDK_DIR=MacOSX-SDKs' --build-arg 'MACOS_SDK_FILE=MacOSX11.3.sdk.tar.xz' .` failed with exit status: 1

Here, it's saying that it is missing the MacOSX-SDKs folder in cross-toolchains/docker, does that folder exist and can docker read it? Does it have unexpected permissions maybe? Check with something like ls -la

frenchtoastbeer commented 8 months ago

That must be the problem, I'll try putting it in different layers. The example section in the OP says where the SDK flie & dir were for this error, and I am pretty sure I also tried placing them in the root of the cross folder. Which folder should I be in when I attempt

cargo build-docker-image x86_64-apple-darwin-cross \
   --build-arg 'MACOS_SDK_DIR=$DIR' \
   --build-arg 'MACOS_SDK_FILE=$FILE'  \
   --tag local

command, and should the file and dir be in the same root? EDIT: (not root, I meant folder I ran the command from)

Emilgardis commented 8 months ago

it should be placed in cross/docker/cross-toolchains/docker

frenchtoastbeer commented 8 months ago

That's very strange then! My environment looks like:

[fr3ncht0ast@fr3ncht0ast-g444 docker]$ pwd
/home/fr3ncht0ast/cross/docker
[fr3ncht0ast@fr3ncht0ast-g444 docker]$ ls -la cross-toolchains/docker | grep MacOSX
-rw-r--r--  1 fr3ncht0ast fr3ncht0ast 51233540 Jan  3 13:14 MacOSX11.3.sdk.tar.xz
drwxr-xr-x 22 fr3ncht0ast fr3ncht0ast     4096 Jan  3 13:13 MacOSX-SDKs
[fr3ncht0ast@fr3ncht0ast-g444 docker]$ unset MACOS_SDK_DIR
unset MACOS_SDK_FILE
unset DIR
unset FILE

and then running the command looks like:

[fr3ncht0ast@fr3ncht0ast-g444 docker]$ cargo build-docker-image x86_64-apple-darwin-cross --build-arg='MACOS_SDK_DIR=MacOSX-SDKs' --build-arg 'MACOS_SDK_FILE=MacOSX11.3.sdk.tar.xz' --tag local
    Finished dev [unoptimized + debuginfo] target(s) in 0.02s
     Running `/home/fr3ncht0ast/cross/target/debug/xtask build-docker-image x86_64-apple-darwin-cross --build-arg=MACOS_SDK_DIR=MacOSX-SDKs --build-arg MACOS_SDK_FILE=MacOSX11.3.sdk.tar.xz --tag local`
[cross] note: Build x86_64-apple-darwin-cross for x86_64-unknown-linux-gnu
[+] Building 3.5s (15/20)                                                                                                                                                                                                                                                                docker-container:xenodochial_wilson
 => [internal] load build definition from Dockerfile.x86_64-apple-darwin-cross                                                                                                                                                                                                                                          0.0s
 => => transferring dockerfile: 1.17kB                                                                                                                                                                                                                                                                                  0.0s
 => [internal] load metadata for docker.io/library/ubuntu:20.04                                                                                                                                                                                                                                                         0.6s
 => [auth] library/ubuntu:pull token for registry-1.docker.io                                                                                                                                                                                                                                                           0.0s
 => [internal] load .dockerignore                                                                                                                                                                                                                                                                                       0.0s
 => => transferring context: 223B                                                                                                                                                                                                                                                                                       0.0s
 => ERROR importing cache manifest from ghcr.io/cross-rs/x86_64-apple-darwin-cross:main                                                                                                                                                                                                                                 1.0s
 => [internal] load build context                                                                                                                                                                                                                                                                                       1.9s
 => => transferring context: 365B                                                                                                                                                                                                                                                                                       1.9s
 => [ 1/13] FROM docker.io/library/ubuntu:20.04@sha256:f2034e7195f61334e6caff6ecf2e965f92d11e888309065da85ff50c617732b8                                                                                                                                                                                                 0.0s
 => => resolve docker.io/library/ubuntu:20.04@sha256:f2034e7195f61334e6caff6ecf2e965f92d11e888309065da85ff50c617732b8                                                                                                                                                                                                   0.0s
 => [auth] cross-rs/x86_64-apple-darwin-cross:pull token for ghcr.io                                                                                                                                                                                                                                                    0.0s
 => CACHED [ 2/13] COPY common.sh lib.sh /                                                                                                                                                                                                                                                                              0.0s
 => CACHED [ 3/13] RUN /common.sh                                                                                                                                                                                                                                                                                       0.0s
 => CACHED [ 4/13] COPY cmake.sh /                                                                                                                                                                                                                                                                                      0.0s
 => CACHED [ 5/13] RUN /cmake.sh                                                                                                                                                                                                                                                                                        0.0s
 => CACHED [ 6/13] COPY xargo.sh /                                                                                                                                                                                                                                                                                      0.0s
 => CACHED [ 7/13] RUN /xargo.sh                                                                                                                                                                                                                                                                                        0.0s
 => ERROR [ 8/13] COPY MacOSX-SDKs/MacOSX11.3.sdk.tar.xz* /                                                                                                                                                                                                                                                             0.0s
------
 > importing cache manifest from ghcr.io/cross-rs/x86_64-apple-darwin-cross:main:
------
------
 > [ 8/13] COPY MacOSX-SDKs/MacOSX11.3.sdk.tar.xz* /:
------
Dockerfile.x86_64-apple-darwin-cross:21
--------------------
  19 |     ARG MACOS_SDK_URL
  20 |     # wildcard workaround so we can copy the file only if it exists
  21 | >>> COPY $MACOS_SDK_DIR/$MACOS_SDK_FILE* /
  22 |     COPY cross-toolchains/docker/darwin.sh /
  23 |     RUN /darwin.sh
--------------------
ERROR: failed to solve: lstat /tmp/buildkit-mount372397504/MacOSX-SDKs: no such file or directory
Error: 
   0: `docker buildx build --platform linux/amd64 --build-arg 'CROSS_TARGET_TRIPLE=X86_64_APPLE_DARWIN_CROSS' --load --pull --cache-from 'type=registry,ref=ghcr.io/cross-rs/x86_64-apple-darwin-cross:main' --tag ghcr.io/cross-rs/x86_64-apple-darwin-cross:local --label 'org.cross-rs.for-cross-target=x86_64-apple-darwin-cross' --label 'org.cross-rs.runs-with=x86_64-unknown-linux-gnu' --file /home/fr3ncht0ast/cross/docker/cross-toolchains/docker/Dockerfile.x86_64-apple-darwin-cross --progress auto --build-arg 'MACOS_SDK_DIR=MacOSX-SDKs' --build-arg 'MACOS_SDK_FILE=MacOSX11.3.sdk.tar.xz' .` failed with exit status: 1

Location:
   xtask/src/build_docker_image.rs:309

Warning: call to docker failed
Suggestion: is `buildx` available for the container engine?
Note: disable the `buildkit` dependency optionally with `CROSS_CONTAINER_ENGINE_NO_BUILDKIT=1`

Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it.
Run with RUST_BACKTRACE=full to include source snippets.

and this fails if I try the docker build-docker-image command from either cross/docker or cross/docker/cross-toolchains/docker and it fails even when I stop all containers and delete all container images inbetween runs. Could the issue potentially with my docker installation?

Emilgardis commented 8 months ago

move the MacOSX11.3.sdk.tar.xz file into the MacOSX-SDKs folder, cwd doesn't matter

frenchtoastbeer commented 8 months ago

I'm not sure what the issue is, that still didn't work. I've pulled the directory out entirely and it will move forward with just the tarball instead. Now it seems to be hitting a new error, perhaps on osxcross's end?

14.50 + git clone https://github.com/tpoechtrager/osxcross --depth 1
14.50 Cloning into 'osxcross'...
15.17 + cd osxcross
15.17 + git fetch --depth=1 origin 17bb5e2d0a46533c1dd525cf4e9a80d88bd9f00e
15.79 From https://github.com/tpoechtrager/osxcross
15.79  * branch            17bb5e2d0a46533c1dd525cf4e9a80d88bd9f00e -> FETCH_HEAD
15.80 + git checkout 17bb5e2d0a46533c1dd525cf4e9a80d88bd9f00e
15.82 Note: switching to '17bb5e2d0a46533c1dd525cf4e9a80d88bd9f00e'.
15.82 
15.82 You are in 'detached HEAD' state. You can look around, make experimental
15.82 changes and commit them, and you can discard any commits you make in this
15.82 state without impacting any branches by switching back to a branch.
15.82 
15.82 If you want to create a new branch to retain commits you create, you may
15.82 do so (now or later) by using -c with the switch command. Example:
15.82 
15.82   git switch -c <new-branch-name>
15.82 
15.82 Or undo this operation with:
15.82 
15.82   git switch -
15.82 
15.82 Turn off this advice by setting config variable advice.detachedHead to false
15.82 
15.82 HEAD is now at 17bb5e2 build_gcc.sh: Update GCC version to 12.1.0
15.82 + [[ -f /MacOSX11.3.sdk.tar.xz ]]
15.82 + pushd tarballs
15.82 + curl --retry 3 -sSfL '' -O
15.82 /tmp/tmp.X1oKLBdtHf/osxcross/tarballs /tmp/tmp.X1oKLBdtHf/osxcross /
15.83 curl: (23) Failed writing received data to disk/application
------
Dockerfile.x86_64-apple-darwin-cross:23
--------------------
  21 |     COPY $MACOS_SDK_DIR/$MACOS_SDK_FILE* /
  22 |     COPY cross-toolchains/docker/darwin.sh /
  23 | >>> RUN /darwin.sh
  24 |     
  25 |     COPY cross-toolchains/docker/darwin-symlink.sh /
--------------------
ERROR: failed to solve: process "/bin/sh -c /darwin.sh" did not complete successfully: exit code: 23
Emilgardis commented 8 months ago

it's now expecting a url to download the tarball, did you provide one according to the readme? https://github.com/cross-rs/cross-toolchains#apple-targets

https://github.com/cross-rs/cross-toolchains/blob/7ce22c54fa650742897f25a1c46d3e0a60edda81/docker/darwin.sh#L47

frenchtoastbeer commented 8 months ago

I've re-read the article you mentioned, and it seems to be saying "... the sdk, which you can then provide to the Docker image as either a local file or download link."

Are you saying the code appears to require a download URL now?

frenchtoastbeer commented 8 months ago

Can confirm the darwin.sh script proceeds with a URL and fails with a local file. I think it might be really helpful to have a full example showing how to get the x86_64-apple-darwin target set up, or at least update the readme to prevent others from making these same mistakes.

Emilgardis commented 8 months ago

Are you saying the code appears to require a download URL now?

I don't know what you did as you didn't show how you called build-docker-image, but the check you hit was this one which is just a check if that file exists or not not, and if it doesn't then it uses a url to download instead.

I'll try this myself to see if there's anything wrong with it but right now I don't see that there is.

frenchtoastbeer commented 8 months ago

I ended up abandoning attempts to use cross-rs/cross-toolchains to do this work, and have build a VM with an older version of OSX for my purposes. Should I close this issue out or would you prefer it remain open?

Emilgardis commented 8 months ago

Ok! No worries, another thing you could do use (ab)use github free tier of actions which should include a osx 11 machine via macos-11