Fix `docker/linux-image.sh` script

gamma0987 commented 7 months ago

As promised here's the fixed linux-image.sh script.

As outlined in #1399 this pr fixes:

Missing gpg key 8D69674688B6CB36 (Debian Ports Archive Automatic Signing Key (2024) ftpmaster@ports-master.debian.org)
The binaries and busybox might also be installed into /usr/bin/ instead of /bin.
The kernel modules might also installed into /usr/lib/modules instead of /lib/modules.
The kernel modules might be compressed with xz and end with *.ko.xz instead of just *.ko

The first commit adds some vim special files to .gitignore and in the second commit the script was formatted with shfmt with standard settings and some minor issues from shellcheck were fixed.

The third commit fixes the actual issues. Not sure if you wanna keep the first two commits but they made working on this file easier.

Closes #1399

Emilgardis commented 7 months ago

/ci try

This is great! You can see the CI run here

Emilgardis commented 7 months ago

ugh, forgot I havn't fixed the try action yet, it skips the actual build of the images :D

gamma0987 commented 7 months ago

I was just about to ask if the script is actually running somewhere :)

Emilgardis commented 7 months ago

/ci try

should be fixed now! https://github.com/cross-rs/cross/actions/runs/7438545066?pr=1405+%28comment%29

Emilgardis commented 7 months ago

The mips*-musl errors are unrelated, should be fixed with https://github.com/cross-rs/cross/pull/1407

You can see the latest CI run here https://github.com/cross-rs/cross/actions/runs/7391006735, basically there's no changes

As seen, the ci still fails, but that's because it's during qemu-user, do you think it would make sense to remove qemu-user from powerpc? That would be done here

github-actions[bot] commented 7 months ago

Diff for comment

gamma0987 commented 7 months ago

basically there's no changes

That's because the ci job didn't pick up the commits and the new linux-image.sh script. I'm not sure what's going wrong but I downloaded the logs of https://github.com/cross-rs/cross/actions/runs/7438545066?pr=1405 and did a grep:

❯ rg 'curl.*2023' .
./try  target (powerpc64-unknown-linux-gnu,ubuntu-latest)/10_Build Docker image.txt
7950:2024-01-07T13:49:01.9909003Z #22 10.43 + curl --retry 3 -sSfL 'https://www.ports.debian.org/archive_{2020,2021,2022,2023}.key' -O

./try  target (riscv64gc-unknown-linux-gnu,ubuntu-latest)/10_Build Docker image.txt
7633:2024-01-07T13:49:07.4924205Z #22 11.40 + curl --retry 3 -sSfL 'https://www.ports.debian.org/archive_{2020,2021,2022,2023}.key' -O

./20_try  target (powerpc64-unknown-linux-gnu,ubuntu-latest).txt
9265:2024-01-07T13:49:01.9909044Z #22 10.43 + curl --retry 3 -sSfL 'https://www.ports.debian.org/archive_{2020,2021,2022,2023}.key' -O

./4_try  target (x86_64-unknown-linux-gnucentos,ubuntu-latest).txt
2457:2024-01-07T13:49:00.4962792Z #14 17.02 + curl --retry 3 -sSfL 'https://www.ports.debian.org/archive_{2020,2021,2022,2023}.key' -O

./try  target (aarch64-unknown-linux-gnucentos,ubuntu-latest)/10_Build Docker image.txt
1185:2024-01-07T13:37:26.7686432Z #15 19.64 + curl --retry 3 -sSfL 'https://www.ports.debian.org/archive_{2020,2021,2022,2023}.key' -O

./62_try  target (aarch64-unknown-linux-gnucentos,ubuntu-lates.txt
2502:2024-01-07T13:37:26.7686467Z #15 19.64 + curl --retry 3 -sSfL 'https://www.ports.debian.org/archive_{2020,2021,2022,2023}.key' -O

./24_try  target (sparc64-unknown-linux-gnu,ubuntu-latest).txt
8965:2024-01-07T13:49:16.5505277Z #22 11.54 + curl --retry 3 -sSfL 'https://www.ports.debian.org/archive_{2020,2021,2022,2023}.key' -O

./try  target (x86_64-unknown-linux-gnucentos,ubuntu-latest)/10_Build Docker image.txt
1140:2024-01-07T13:49:00.4962754Z #14 17.02 + curl --retry 3 -sSfL 'https://www.ports.debian.org/archive_{2020,2021,2022,2023}.key' -O

./try  target (sparc64-unknown-linux-gnu,ubuntu-latest)/10_Build Docker image.txt
7648:2024-01-07T13:49:16.5505246Z #22 11.54 + curl --retry 3 -sSfL 'https://www.ports.debian.org/archive_{2020,2021,2022,2023}.key' -O

./try  target (powerpc-unknown-linux-gnu,ubuntu-latest)/10_Build Docker image.txt
7811:2024-01-07T13:49:18.8106413Z #22 10.77 + curl --retry 3 -sSfL 'https://www.ports.debian.org/archive_{2020,2021,2022,2023}.key' -O

./19_try  target (powerpc-unknown-linux-gnu,ubuntu-latest).txt
9122:2024-01-07T13:49:18.8106437Z #22 10.77 + curl --retry 3 -sSfL 'https://www.ports.debian.org/archive_{2020,2021,2022,2023}.key' -O

./22_try  target (riscv64gc-unknown-linux-gnu,ubuntu-latest).txt
8945:2024-01-07T13:49:07.4924240Z #22 11.40 + curl --retry 3 -sSfL 'https://www.ports.debian.org/archive_{2020,2021,2022,2023}.key' -O

If this would have been the new linux-image.sh script this line should include the 2024 key:

curl --retry 3 -sSfL 'https://www.ports.debian.org/archive_{2020,2021,2022,2023,2024}.key' -O

gamma0987 commented 7 months ago

Looks like the error already happens in the actions/checkout@v3 step

0s
Run actions/checkout@v3
Syncing repository: cross-rs/cross
Getting Git version info
Temporarily overriding HOME='/home/runner/work/_temp/9e0762cf-65f4-49d3-835d-a4d525697544' before making global git config changes
Adding repository directory to the temporary git global config as a safe directory
/usr/bin/git config --global --add safe.directory /home/runner/work/cross/cross
Deleting the contents of '/home/runner/work/cross/cross'
Initializing the repository
Disabling automatic garbage collection
Setting up auth
Fetching the repository
Determining the checkout info
Checking out the ref
  /usr/bin/git checkout --progress --force -B main refs/remotes/origin/main
  Switched to a new branch 'main'
  branch 'main' set up to track 'origin/main'.
/usr/bin/git log -[1](https://github.com/cross-rs/cross/actions/runs/7438545066/job/20237413186?pr=1405#step:3:1) --format='%H'
'4da645bc4553fbe6b16df320d7c5c9cc6839dba8'

because it checks out the main branch instead of the pull request.

I've rebased onto the main branch to pick up the latest ci changes from #1406. Maybe that helps...

Emilgardis commented 7 months ago

hmm, thank you for looking into it @gamma0987, with that information I know what's wrong. I'll fix it

gamma0987 commented 7 months ago

Sure. Thanks for fixing it

Emilgardis commented 7 months ago

Now it should work!

/ci try

ehm, when https://github.com/cross-rs/cross/pull/1408 is merged, did the comment a bit prematurely :)

Emilgardis commented 7 months ago

/ci try

https://github.com/cross-rs/cross/actions/runs/7444481056?pr=1405+%28comment%29

github-actions[bot] commented 7 months ago

Diff for comment

gamma0987 commented 7 months ago

Looks like it worked so far. Any idea why the zig job fails? The riscv64gc-unknown-linux-gnu target doesn't build, too. Since there's something wrong in the linux-image.sh script, I can try fixing it.

gamma0987 commented 7 months ago

I tried locally and the riscv64gc-unknown-linux-gnu target should build and run now. The zig job doesn't use the linux-image.sh script, so I think this failure is unrelated to the changes of this pr.

Could you please trigger a last test run?

Emilgardis commented 7 months ago

/ci try

Emilgardis commented 7 months ago

Diff for comment

Emilgardis commented 7 months ago

Oops, trying out different ways to represent the data and forgot to remove the command that actually makes the comment :D

github-actions[bot] commented 7 months ago

Diff for comment

gamma0987 commented 7 months ago

The ci test for riscv64 fails with a linker error unsupported ISA subset 'z'. That's where my knowledge about risc ends. Do you have any idea? If not, I have to leave this to someone else and remove the last commit.

Emilgardis commented 7 months ago

I think we can keep the commit anyway, we'll make fixes for the remaining targets

gamma0987 commented 7 months ago

Sure :)

cross-rs / cross

Fix `docker/linux-image.sh` script #1405