Open szlend opened 2 months ago
Thank you for the report.
I agree with your proposal and contributions would be welcome ;)
After reviewing the source code, I can see that this is an issue all across the board and not just isolated to buildImage
. I feel like we could restructure this a bit to make cross-compilation support easier to manage.
We could do the following:
nix2container
with makeScopeWithSplicing'
. This gives us a way to express different platform variants of our builders/derivations (e.g. buildHost
, hostTarget
, etc).callPackage
pattern. This enables package splicing so nativeBuildInputs
/buildInputs
automatically pick the correct platform variants of our derivations when cross-compiling.callPackage
pattern can be pretty noisy, I would suggest extracting individual builders/derivations to their own nix files where it makes sense. Similar to how crane does it here: https://github.com/ipetkov/crane/tree/master/libFor reference, I implemented proper splicing/cross-compilation support in crane recently: https://github.com/ipetkov/crane/pull/652
The alternative approach would be reviewing the source code and replacing certain instances of pkgs.<package>
with pkgs.pkgsBuildHost.<package>
and manually defining pkgsBuildHost
variants of nix2container-bin
/skopeo-nix2container
. This is a more straightforward approach, but at the cost of a lot of cross-compilation specific noise.
@nlewo Which approach would you prefer? Both of these would be fully backwards compatible. I think the splicing approach would be nicer to work with, but it would be a larger change, cause big merge conflicts with open PRs and more difficult to review. Though the project has a pretty decent test coverage so we can be fairly certain we haven't broken something.
I opened a PoC PR in https://github.com/nlewo/nix2container/issues/139 to show what I mean.
It's currently not possible to cross-build images for a foreign architecture. Using
nix2container
onnixpkgs
with a foreigncrossSystem
will attempt to build thenix2container-bin
andjq
binaries for the foreign system and then try to execute them at build time. You can work around this issue by usingnix2container-bin
based onpkgs.pkgsBuildHost
. However this doesn't work correctly either, becausearch
is hardcoded toruntime.GOARCH
.You can work around both of these issues with a small wrapper:
Ideally this workaround wouldn't be necessary. We could fix this by:
nix2container-bin
based onpkgs.pkgsBuildHost
(this is equivalent topkgs
whencrossSystem
is not set).buildImage
pass a (new)--arch
argument tonix2container-bin
based onpkgs.go.GOARCH
.