docker pull should have a --verbose option to see all of the network calls

jefflage commented 7 years ago

Many organizations are now applying more aggressive outbound blocking of network requests, every to the point of blocking all outbound traffic unless specifically whitelisted. It would be good if there were a command line flag available on docker pull such that you can force it to print out the host/ip and port of any outbound requests. This would make it easier to work with security teams to get all of the necessary destinations correctly whitelisted.

dnephin commented 7 years ago

Wouldn't tcpdump, wireshark, etc be a lot more reliable and the standard way of identifying these requests?

thaJeztah commented 7 years ago

I don't think this should be a command-line option; the client doesn't pull the image, it's the daemon doing so. So if configuration changes are needed, those likely need to be made on the daemon host.

For the daemon host, there is already an option to obtain this information; enable debug mode on dockerd (start dockerd --debug or set "debug": true in the daemon.json configuration), and the requests are printed in the daemon logs;

Here's what's shown if both --disable-legacy-registry=false and --insecure-registry=some-domain.example.com is set;

docker pull some-domain.example.com/foobar:latest

DEBU[0011] Calling POST /v1.30/images/create?fromImage=some-domain.example.com%2Ffoobar&tag=latest
DEBU[0011] Trying to pull some-domain.example.com/foobar from https://some-domain.example.com v2
WARN[0011] Error getting v2 registry: Get https://some-domain.example.com/v2/: dial tcp some-domain.example.com:443: getsockopt: connection refused
INFO[0011] Attempting next endpoint for pull after error: Get https://some-domain.example.com/v2/: dial tcp some-domain.example.com:443: getsockopt: connection refused
DEBU[0011] Trying to pull some-domain.example.com/foobar from http://some-domain.example.com v2
INFO[0011] Attempting next endpoint for pull after error: error parsing HTTP 404 response body: invalid character '<' looking for beginning of value: "<html>\r\n<head><title>404 Not Found</title></head>\r\n<body bgcolor=\"white\">\r\n<center><h1>404 Not Found</h1></center>\r\n<hr><center>nginx/1.13.3</center>\r\n</body>\r\n</html>\r\n"
DEBU[0011] Trying to pull some-domain.example.com/foobar from https://some-domain.example.com v1
DEBU[0011] attempting v1 ping for registry endpoint https://some-domain.example.com/v1/
DEBU[0011] Fallback from error: Get https://some-domain.example.com/v1/_ping: dial tcp some-domain.example.com:443: getsockopt: connection refused
INFO[0011] Attempting next endpoint for pull after error: Get https://some-domain.example.com/v1/_ping: dial tcp some-domain.example.com:443: getsockopt: connection refused
DEBU[0011] Trying to pull some-domain.example.com/foobar from http://some-domain.example.com v1
DEBU[0011] [registry] Calling GET http://some-domain.example.com/v1/repositories/foobar/images
ERRO[0012] Not continuing with pull after error: Error: image foobar:latest not found

Does that solve your use case?

jefflage commented 7 years ago

If this option will show all of the hosts that are communicated with in the course of resolving and retrieving an image, then yes it will.

Alternately, if there is admin documentation somewhere that lists all domains that would need to be whitelisted for docker to work effectively behind a restrictive firewall, that would also work.

thaJeztah commented 7 years ago

I'm not sure such a list exists; the layers themselves may be distributed through different hosts, based on the location that the pull originates from. However, the best source of information (for Docker Hub) would be support@docker.com - I don't have that information available 😅

foxundermoon commented 6 years ago

+1

s50600822 commented 5 years ago

+1 See my case where I needed it on https://github.com/docker/for-mac/issues/1878's comment. I actually need to see the request header to verify why the Java DockerClient fails to auth in a few edge cases.

PardhuMadipalli commented 4 years ago

I find this is a very useful feature if implemented. I faced some issues where I could login to docker registry, but the image pull fails, although I could pull the image from other nodes. The error message says I don't have a pull access or the repository does not exist. I could not figure what the actual issue is.

ag-TJNII commented 4 years ago

enable debug mode on dockerd (start dockerd --debug or set "debug": true in the daemon.json configuration), and the requests are printed in the daemon logs

I'm not seeing this behavior. With debug logging on I'm only seeing generic "pulling" messages but not the actual API calls. I'm troubleshooting registry behavior that doesn't match the documentation so I do need to see the exact API calls. Documentation on how to enable debug at this level gets a bit hard to follow, is there a log level higher than "debug"?

sebastian-philipp commented 3 years ago

with the reduced rate limit, this is even more annoying: We now have mandatory local proxy registries and having no simple way to verify this is bad.

JiangHongTiao commented 3 years ago

Hi guys, I would need also a bit more detailed output. I'm trying to pull image from our local artifactory, it seems the image is downloaded, however then I get error: Unknown blob.

My output of pull:

docker pull company.com/product/images/postgres12:empty-1.2
empty-1.2: Pulling from product/images/postgres12
89d9c30c1d48: Pulling fs layer
c7bb53530e14: Pulling fs layer
87d24f6e3877: Pulling fs layer
623dc2a6d06d: Pulling fs layer
c202de0fa856: Pulling fs layer
b74a7f2415c8: Pulling fs layer
cf53f01279c5: Pulling fs layer
b12915506035: Pulling fs layer
32b47b2ae54e: Pulling fs layer
19765a6f2d9d: Pulling fs layer
223b2b3ee6d3: Pulling fs layer
623dc2a6d06d: Waiting
b12915506035: Waiting
c202de0fa856: Waiting
b74a7f2415c8: Waiting
cf53f01279c5: Waiting
19765a6f2d9d: Waiting
223b2b3ee6d3: Waiting
32b47b2ae54e: Waiting
c7bb53530e14: Verifying Checksum
c7bb53530e14: Download complete
87d24f6e3877: Verifying Checksum
87d24f6e3877: Download complete
89d9c30c1d48: Downloading
c202de0fa856: Download complete
cf53f01279c5: Verifying Checksum
cf53f01279c5: Download complete
b74a7f2415c8: Verifying Checksum
b74a7f2415c8: Download complete
b12915506035: Verifying Checksum
b12915506035: Download complete
19765a6f2d9d: Verifying Checksum
19765a6f2d9d: Download complete
32b47b2ae54e: Verifying Checksum
32b47b2ae54e: Download complete
223b2b3ee6d3: Verifying Checksum
223b2b3ee6d3: Download complete
623dc2a6d06d: Verifying Checksum
623dc2a6d06d: Download complete
unknown blob

Is there any way how can I get more detailed output? My daemon.json looks like:

{
  "registry-mirrors": [],
  "insecure-registries": [],
  "debug": true,
  "experimental": false,
  "features": {
    "buildkit": true
  },
  "builder": {
    "gc": {
      "enabled": true,
      "defaultKeepStorage": "20GB"
    }
  }
}

thaJeztah commented 3 years ago

Look in the logs for the daemon, they may contain more details about the failure. "unknown blob" sounds like a possible issue with the image in the registry though

JiangHongTiao commented 3 years ago

sorry for late response. Yes, there was probably a bug in our artifactory, our IT has applied some workaround and it has started working.

jokester commented 2 years ago

An improper workaround is to write a yaml and docker compose pull the image. Sometimes this gives better message than docker pull

aejnsn commented 1 year ago

@thaJeztah That doesn't show the entirety of the request's content. I want to diagnose a container registry and behavior of the CLI, I have yet to find a way to view request or response data for pulling an image.

kudodenko commented 1 year ago

+1

cloudwitch commented 1 year ago

We have an issue where we're getting EOF errors and have been unable to recreate the issue anywhere but our Jenkins worker nodes when running pipelines. It's a intermittent issue, so we would love to be able to have extra debugging enabled until we figure it out.

It's a weird one, from what we can tell, nothing changed in the environment, and is unable to be replicated anywhere reliably.

QGB commented 7 months ago

root@ubuntu:/home/qgb# docker pull manhinhang/futu-opend-docker:ubuntu-8.0.4008
ubuntu-8.0.4008: Pulling from manhinhang/futu-opend-docker
58690f9b18fc: Already exists 
b51569e7c507: Already exists 
da8ef40b9eca: Already exists 
fb15d46c38dc: Already exists 
d42fab5f7fe5: Pulling fs layer

hangs

la3rence commented 5 months ago

I'am also wanna a -v like kubectl does. But we can check the log files instead: https://stackoverflow.com/questions/30969435/where-is-the-docker-daemon-log

docker / cli

docker pull should have a --verbose option to see all of the network calls #336