Expand support for musl-based containers

baronfel commented 1 year ago

As of 0.3.2 if a user targets a musl-based container (e.g. mcr.microsoft.com/dotnet/aspnet:7.0-alpine) they cannot use a musl-based RID (e.g. linux-musl-x64) to ensure the correct assets get built for that runtime. We present them an error saying that only linux-x64, linux-arm, and linux-arm64 RIDs are valid for that image. This is because the RIDs we generate for comparison purposes based on the manifest list for the 7.0-alpine tag of that image do not take musl into account. If the user follows our error messages and uses a non-musl Linux RID, the generated container image will not launch because the entrypoint (/app/<projectname>) is not launchable by the shell due to the libc incompatibility.

There is no data point on the manifest list or the individual manifests that would tell us that the image is musl based, so we can't change our RID-comparison generation to take this into account. The best guess we have could be to look for alpine in the image tag name, but that feels brittle and incomplete.

The user's only out right now is to explicitly specify a more detailed image tag name. In the example above, the user would change from mcr.microsoft.com/dotnet/aspnet:7.0-alpine to mcr.microsoft.com/dotnet/aspnet:7.0-alpine-amd64.

We should document this caveat and consult with @mthalman and other container gurus to see if there is any data point we can drive inference logic from.

Resources

manifest list for the 7.0-alpine tag of the aspnet image

```json { "schemaVersion": 2, "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json", "manifests": [ { "mediaType": "application/vnd.docker.distribution.manifest.v2+json", "size": 951, "digest": "sha256:64989a860993e5fea955998d5c347078ff1424c1a23e749c5c6efe3d7f6eb38c", "platform": { "architecture": "amd64", "os": "linux" } }, { "mediaType": "application/vnd.docker.distribution.manifest.v2+json", "size": 951, "digest": "sha256:b9b1d0e452fd224ca574cbb3b8bf5c15628c9ad861540fd7ed45504f8fc309b4", "platform": { "architecture": "arm", "os": "linux", "variant": "v7" } }, { "mediaType": "application/vnd.docker.distribution.manifest.v2+json", "size": 951, "digest": "sha256:c267c2bd636bbe33f00b787cc2fed236108c6a266dc98870b08c842521ae585c", "platform": { "architecture": "arm64", "os": "linux", "variant": "v8" } } ] } ```

manifest for the amd64 variant in that same list

```json { "architecture": "amd64", "config": { "Hostname": "", "Domainname": "", "User": "", "AttachStdin": false, "AttachStdout": false, "AttachStderr": false, "Tty": false, "OpenStdin": false, "StdinOnce": false, "Env": [ "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin", "ASPNETCORE_URLS=http://+:80", "DOTNET_RUNNING_IN_CONTAINER=true", "DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=true", "DOTNET_VERSION=7.0.2" ], "Cmd": [ "/bin/sh" ], "Image": "sha256:f1c79b6a9bc29dde77de65e8f7a0ae863ac36582e4f0cfe9b0ee146fcc98b956", "Volumes": null, "WorkingDir": "", "Entrypoint": null, "OnBuild": null, "Labels": null }, "container": "89100c5750b8abf5f8888ed0406ff72258051edc31702e5921597f97a63ee1f5", "container_config": { "Hostname": "", "Domainname": "", "User": "", "AttachStdin": false, "AttachStdout": false, "AttachStderr": false, "Tty": false, "OpenStdin": false, "StdinOnce": false, "Env": [ "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin", "ASPNETCORE_URLS=http://+:80", "DOTNET_RUNNING_IN_CONTAINER=true", "DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=true", "DOTNET_VERSION=7.0.2" ], "Cmd": [ "/bin/sh", "-c", "wget -O dotnet.tar.gz https://dotnetcli.azureedge.net/dotnet/Runtime/$DOTNET_VERSION/dotnet-runtime-$DOTNET_VERSION-linux-musl-x64.tar.gz && dotnet_sha512='83f50faf95a2ba3756da838fe2f3272f18494f5707aa787e6519f2b145107297c20a974221b5a4ef383dc92486c43cf6899f51a8e776c7ed7bcf1855ea2aba15' && echo \"$dotnet_sha512 dotnet.tar.gz\" | sha512sum -c - && mkdir -p /usr/share/dotnet && tar -oxzf dotnet.tar.gz -C /usr/share/dotnet && rm dotnet.tar.gz && ln -s /usr/share/dotnet/dotnet /usr/bin/dotnet" ], "Image": "sha256:f1c79b6a9bc29dde77de65e8f7a0ae863ac36582e4f0cfe9b0ee146fcc98b956", "Volumes": null, "WorkingDir": "", "Entrypoint": null, "OnBuild": null, "Labels": null }, "created": "2023-01-10T14:50:37.904874447Z", "docker_version": "20.10.17", "history": [ { "created": "2023-01-09T17:05:20.497231175Z", "created_by": "/bin/sh -c #(nop) ADD file:e4d600fc4c9c293efe360be7b30ee96579925d1b4634c94332e2ec73f7d8eca1 in / " }, { "created": "2023-01-09T17:05:20.656498283Z", "created_by": "/bin/sh -c #(nop) CMD [\"/bin/sh\"]", "empty_layer": true }, { "created": "2023-01-10T14:50:28.253977868Z", "created_by": "/bin/sh -c apk add --no-cache ca-certificates krb5-libs libgcc libintl libssl3 libstdc++ zlib" }, { "created": "2023-01-10T14:50:29.134673936Z", "created_by": "/bin/sh -c #(nop) ENV ASPNETCORE_URLS=http://+:80 DOTNET_RUNNING_IN_CONTAINER=true DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=true", "empty_layer": true }, { "created": "2023-01-10T14:50:34.279194802Z", "created_by": "/bin/sh -c #(nop) ENV DOTNET_VERSION=7.0.2", "empty_layer": true }, { "created": "2023-01-10T14:50:37.904874447Z", "created_by": "/bin/sh -c wget -O dotnet.tar.gz https://dotnetcli.azureedge.net/dotnet/Runtime/$DOTNET_VERSION/dotnet-runtime-$DOTNET_VERSION-linux-musl-x64.tar.gz && dotnet_sha512='83f50faf95a2ba3756da838fe2f3272f18494f5707aa787e6519f2b145107297c20a974221b5a4ef383dc92486c43cf6899f51a8e776c7ed7bcf1855ea2aba15' && echo \"$dotnet_sha512 dotnet.tar.gz\" | sha512sum -c - && mkdir -p /usr/share/dotnet && tar -oxzf dotnet.tar.gz -C /usr/share/dotnet && rm dotnet.tar.gz && ln -s /usr/share/dotnet/dotnet /usr/bin/dotnet" } ], "os": "linux", "rootfs": { "type": "layers", "diff_ids": [ "sha256:8e012198eea15b2554b07014081c85fec4967a1b9cc4b65bd9a4bce3ae1c0c88", "sha256:cc803a0de68f119ef7c7b82697ca041ced88a9474df6e2fcf93dc3eafeb4313c", "sha256:cd7d4badb0e49eccd62ebd4cab6729f7b9283e9d4fc768a0cc01b38e93ddc2ef" ] } } ```

mthalman commented 1 year ago

This may be a good case for the usage of annotations. They allow for metadata to be attached to various registry objects. I'm waiting to hear back from the MAR team on the status of annotation support.

baronfel commented 1 year ago

I have conflicting thoughts about the use of annotations:

1) Awesome, that would be easy for us to consume in this tooling! 2) Awesome, we can give a good experience to users using the MS .NET base images! 3) Crap, guess this isn't something the overall container ecosystem has solved yet? 4) Crap, that means we'll need to think long and hard about how to warn users about potential RID conflicts. We'd need to detect that the reference signaled musl use in some way (possibly just checking the image reference for 'alpine'?), then change the way we generate our synthetic RID graph for compatibility checking to use the linux-musl RID variants instead of just 'linux'.

baronfel commented 1 year ago

This will likely depend on #401 a lot to get done. We should probably take the ContainerRuntimeIdentifier into account during inference and image selection - since musl-ness can't be inferred from the image metadata we should make the following set of adjustments:

if the ContainerBaseImage is being inferred AND the ContainerRuntimeIdentifier is a musl RID AND no ContainerFamily has been provided, then the ContainerFamily should be inferred to be alpine for the MS containers
if the ContainerBaseImage is explicitly set, AND the parsed Repository name of that image is alpine or library/alpine OR the parsed Tag of the ContainerBaseImage contains alpine, AND the ContainerRuntimeIdentifier is being inferred, then the ContainerRuntimeIdentifier should be set to a musl variant

No change is required during manifest list selection, because all of the linux-musl-ARCH RID variants inherit from the linux-ARCH variant, so the RIDs that we infer for manifests are kinda-sorta-correct already.

baronfel commented 1 year ago

ContainerFamily is implemented, so the pathway is clear for using a musl-rid as a flag to support this more easily. Folks that want to easily do this now can set the following if they want to be always-correct until we implement this in the tooling:

<PropertyGroup>
  <ContainerFamily Condition="'$(ContainerFamily)' == '' and '$(RuntimeIdentifier)' != '' and $(RuntimeIdentifier.Contains('-musl-'))">alpine</ContainerFamily>
</PropertyGroup>

richlander commented 10 months ago

I just ran into this while working on some demos.

baronfel commented 8 months ago

With 8.0.200, targeting musl RIDs will use the alpine variants automatically!

dotnet / sdk-container-builds

Expand support for musl-based containers #301

Resources