debian bookworm `apt-get update` fails with `fatal error in libgcrypt: requested algo not in md context` on host with FIPS mode enabled

[vergenzt]

Related mailing list discussion: Bug#1014517: apt - Fails in FIPS mode in libgcrypt

I originally filed this at https://github.com/docker-library/python/issues/855, but filing here as it seems to be closer to the source of the issue.

Any ideas for how to resolve this?

Repro:

$ sysctl crypto.fips_enabled
crypto.fips_enabled = 1

$ docker pull debian:bookworm && docker run debian:bookworm apt-get update
bookworm: Pulling from library/debian
Digest: sha256:3d868b5eb908155f3784317b3dda2941df87bbbbaa4608f84881de66d9bb297b
Status: Image is up to date for debian:bookworm
docker.io/library/debian:bookworm
Get:1 http://deb.debian.org/debian bookworm InRelease [151 kB]
Get:2 http://deb.debian.org/debian bookworm-updates InRelease [52.1 kB]
Get:3 http://deb.debian.org/debian-security bookworm-security InRelease [48.0 kB]
Get:4 http://deb.debian.org/debian bookworm/main amd64 Packages [8906 kB]
Get:5 http://deb.debian.org/debian bookworm-updates/main amd64 Packages [4732 B]
Get:6 http://deb.debian.org/debian-security bookworm-security/main amd64 Packages [48.0 kB]
Fetched 9210 kB in 1s (6526 kB/s)
Reading package lists...fatal error in libgcrypt, file ../../src/misc.c, line 92, function _gcry_fatal_error: requested algo not in md context

Fatal error: requested algo not in md context

$ docker pull debian:bookworm-slim && docker run debian:bookworm-slim apt-get update
bookworm-slim: Pulling from library/debian
Digest: sha256:9bd077d2f77c754f4f7f5ee9e6ded9ff1dff92c6dce877754da21b917c122c77
Status: Image is up to date for debian:bookworm-slim
docker.io/library/debian:bookworm-slim
Get:1 http://deb.debian.org/debian bookworm InRelease [151 kB]
Get:2 http://deb.debian.org/debian bookworm-updates InRelease [52.1 kB]
Get:3 http://deb.debian.org/debian-security bookworm-security InRelease [48.0 kB]
Get:4 http://deb.debian.org/debian bookworm/main amd64 Packages [8906 kB]
Get:5 http://deb.debian.org/debian bookworm-updates/main amd64 Packages [4732 B]
Get:6 http://deb.debian.org/debian-security bookworm-security/main amd64 Packages [48.0 kB]
Fetched 9210 kB in 1s (6805 kB/s)
Reading package lists...fatal error in libgcrypt, file ../../src/misc.c, line 92, function _gcry_fatal_error: requested algo not in md context

Fatal error: requested algo not in md context

Doesn't seem to affect older versions of debian:

$ docker pull debian:bullseye && docker run debian:bullseye apt-get update
bullseye: Pulling from library/debian
Digest: sha256:a648e10e02af129706b1fb89e1ac9694ae3db7f2b8439aa906321e68cc281bc0
Status: Image is up to date for debian:bullseye
docker.io/library/debian:bullseye
Get:1 http://deb.debian.org/debian bullseye InRelease [116 kB]
Get:2 http://deb.debian.org/debian-security bullseye-security InRelease [48.4 kB]
Get:3 http://deb.debian.org/debian bullseye-updates InRelease [44.1 kB]
Get:4 http://deb.debian.org/debian bullseye/main amd64 Packages [8183 kB]
Get:5 http://deb.debian.org/debian-security bullseye-security/main amd64 Packages [252 kB]
Get:6 http://deb.debian.org/debian bullseye-updates/main amd64 Packages [14.8 kB]
Fetched 8658 kB in 2s (5251 kB/s)
Reading package lists...

$ docker pull debian:buster && docker run debian:buster apt-get update
buster: Pulling from library/debian
Digest: sha256:c21dbb23d41cb3f1c1a7f841e8642bf713934fb4dc5187979bd46f0b4b488616
Status: Image is up to date for debian:buster
docker.io/library/debian:buster
Get:1 http://deb.debian.org/debian buster InRelease [122 kB]
Get:2 http://deb.debian.org/debian-security buster/updates InRelease [34.8 kB]
Get:3 http://deb.debian.org/debian buster-updates InRelease [56.6 kB]
Get:4 http://deb.debian.org/debian buster/main amd64 Packages [7909 kB]
Get:5 http://deb.debian.org/debian-security buster/updates/main amd64 Packages [545 kB]
Get:6 http://deb.debian.org/debian buster-updates/main amd64 Packages [8788 B]
Fetched 8676 kB in 2s (5771 kB/s)
Reading package lists...

[tianon]

If I'm reading it correctly, I think the conclusion in https://bugs.debian.org/1014517 is that Debian does not support (and hasn't ever supported) FIPS mode? :confused:

(I'm not an APT maintainer, and I think https://bugs.debian.org/1014517 really is a more appropriate place to discuss/track this. I'm not really seeing anything we could change in the way we generate/package the rootfs tarballs of the pure Debian upstream builds that might affect this in any meaningful way.)

[vergenzt]

Any idea why the issue only cropped up in the Docker images recently? (~4 days ago) IIUC I think my team's Docker builds have been succeeding under a FIPS-enabled host system before that point.

Currently thinking I'll just work around this by using an older version of Debian, but I'm curious to figure out why it only recently started impacting builds if it's supposedly never been supported. 🤔

[tianon]

The Debian images haven't changed in any way recently (even the last rebuild was ~3 weeks ago). :sweat_smile:

[tianon]

You could try going through https://hub.docker.com/_/debian/tags?name=bookworm-20 as a rough way to travel back in time and try a bunch of older versions of the bookworm image to try and narrow down where it was working.

$ crane ls debian | grep -E '^bookworm-[0-9]+$' | sort -r | head
bookworm-20230703
bookworm-20230612
bookworm-20230522
bookworm-20230502
bookworm-20230411
bookworm-20230320
bookworm-20230227
bookworm-20230208
bookworm-20230202
bookworm-20230109

[tianon]

I'm going to go ahead and close this per the discussion in https://bugs.debian.org/1014517 -- looks like there's more work to do upstream before "FIPS mode enabled Debian" can be a thing. :see_no_evil: