GoogleContainerTools / distroless

🥑 Language focused docker images, minus the operating system.
Apache License 2.0
18.77k stars 1.15k forks source link

Unable to reach certain IP addresses #1679

Open aeneasr opened 1 week ago

aeneasr commented 1 week ago

Describe the bug

We are observing that wget (and our Go services) hangs itself trying to connect to certain IP addresses. Theses addresses resolve fine outside of the distroless container.

In container

% docker run --rm -it gcr.io/distroless/base-debian12:debug
/ #  wget -S https://sts.nih.gov/.well-known/openid-configuration -O -
Connecting to sts.nih.gov (128.231.243.251:443)
wget: note: TLS certificate validation not implemented

<HANGS>

Outside container

% wget -S https://sts.nih.gov/.well-known/openid-configuration -O -
--2024-09-20 18:40:53--  https://sts.nih.gov/.well-known/openid-configuration
Resolving sts.nih.gov (sts.nih.gov)... 128.231.243.251
Connecting to sts.nih.gov (sts.nih.gov)|128.231.243.251|:443... connected.
HTTP request sent, awaiting response... 
  HTTP/1.1 200 

...

To Reproduce

See above. We observe this on multiple platforms, in all networks / devices and all regions.

Expected behavior

Connection should not hang. We believe this broke recently.

Console Output

See above.

loosebazooka commented 1 week ago

Can you try again with new images?

aeneasr commented 1 week ago

Still an issue:

% docker pull  gcr.io/distroless/base-debian12:debug
debug: Pulling from distroless/base-debian12
50a935fa04e7: Already exists 
e1213c3d2d82: Already exists 
9aee425378d2: Already exists 
d2542ff9a028: Already exists 
Digest: sha256:662eaa2606087124ac1fc108291ecad341f6376ce6fa28ac7e1655ec76c6e6d9
Status: Downloaded newer image for gcr.io/distroless/base-debian12:debug
gcr.io/distroless/base-debian12:debug

What's next:
    View a summary of image vulnerabilities and recommendations → docker scout quickview gcr.io/distroless/base-debian12:debug

% docker run --rm -it gcr.io/distroless/base-debian12:debug
/ # 
/ # 
/ # 
/ # 
/ #  wget -S https://sts.nih.gov/.well-known/openid-configuration -O -
Connecting to sts.nih.gov (128.231.243.251:443)
wget: note: TLS certificate validation not implemented
loosebazooka commented 1 week ago

I believe this could just be an issue from wget on busybox (https://github.com/docker-library/busybox/issues/80). What is your goal here? If you want to use the debug image to do something of value, then you're probably better served by a more full featured image.

aeneasr commented 1 week ago

The issue is that it affects all tcp traffic. In our case we have a service written in Go which is trying to reach this host and the http.Do call hangs. We believe that wget and Go hang for the same reason.

So essentially this issue prevents the image from calling specific hosts, and it just blocks the connection which never terminates / times out.

We believe it's an SSL issue, but are not sure.