Docker Pull Limit reached when scanning a locally saved Docker Image

[kiranravindran90]

I have installed TERN & it seems to be working fine for the sample files provided. I have some Docker images locally that I have pulled earlier & want to scan them using TERN. I used tern report -i : command. But getting an error "Docker Pull Limit reached".

• I don't understand why as these images are locally present.
• Just after few secs if I try to Pull images using Docker Pull command, the Pull works. So not sure how the Limit is reached.

It seems to happen when Skopeo is working its craft & trying to pull images.

[kiranravindran90]

Also when it runs & doesnt give the above error, it throws this below error. 2022-07-04 14:56:19,947 - ERROR - skopeo - Error when downloading image: "b'time="2022-07-04T14:56:19+05:30" level=fatal msg="Error initializing source docker://"path of the docker image of the private repo": error pinging.

Not sure why. These images are already pulled from a private repo & available locally.

Note: TERN was installed through Python Virtual Env.

[rnjudge]

@kiranravindran90 can you tell me a little more about the version of Docker that you have installed?

[kiranravindran90]

@rnjudge Docker version 20.10.17, build 100c701.

Is it actually necessary for Skopeo to download or Pull image if image is already available locally?

[rnjudge]

It's not necessary, but Skopeo will pull all images even if they are local, unfortunately. Skoepo doesn't have the concept of checking the local cache for already downloaded images like docker does. If you want to run Tern on a local image, you can docker save the image and use the raw command line option. Using this option will eliminate the pulling of a local image.

$ docker save image:tag > image_name.tar
$ tern report -w image_name.tar

[nicorikken]

I too noticed this behavior, trying to analyze it in a pipeline before pushing it to the registry. I was even considering adding the Tern output to the final image. The docker save is an outcome, but changes the the working of Tern (or its dependencies like Skopeo) would be welcome. I'll try the docker save approach in the meantime. It is worth noting that docker save has the -o flag to write to a file directly, so piping to a file isn't needed: docker save -o image_name.tar image:tag

[kiranravindran90]

Thank you all for the suggestions. As of now yes using it with "docker save" work around. @nicorikken If you find a way to directly scan then please do post an update, will make things easier for me.

[xxLiuxx]

@rnjudge Hi, I saved the local image as .tar file and did a scan. It looks like only the docker environment packages are captured, the packages used in the application are all ignored. In the report, there are some unrecognized commands. E.g. info: Layer created by commands: COPY build/libs/DBServices*.jar /opt/apache-tomcat-7.0.70/app.jar warning: Unrecognized Commands:COPY build/libs/DBServices*.jar /opt/apache-tomcat-7.0.70/app.jar # buildkit

[rnjudge]

@xxLiuxx @kiranravindran90 Things I learened.. Skopeo has support for local images! (TY Ruivalim!) There is a PR that will be included in the next release that I think should add the support you are looking for: https://github.com/tern-tools/tern/pull/1191

[rnjudge]

@xxLiuxx @kiranravindran90 are either of you able to see if you're able to reproduce this issue in the latest release 2.11.0?

[amallayev]

@rnjudge I can confirm that skopeo now pulls local image when specified:

# skopeo --version
skopeo version 1.4.1

# tern --version
Tern version 2.11.0
   python version = 3.10.6 (main, Nov 14 2022, 16:10:14)

# podman --version
podman version 3.4.4

# docker images |grep node
Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg.
docker.io/bitnami/node                                       18.12.1                      ac204693e94f  2 months ago   729 MB

# ./tern report -i docker-daemon:node:18.12.1 -o bitnami-node.18.12.1-prod.tern.$(date +"%d.%m.%Y").txt
2023-01-19 09:43:14,829 - DEBUG - __main__ - Starting...
2023-01-19 09:43:14,829 - DEBUG - prep - Setting up...
2023-01-19 09:43:18,123 - DEBUG - rootfs - Running command: chmod +x /root/ternenv/lib/python3.10/site-packages/tern/tools/fs_hash.sh2023-01-19 09:43:18,125 - DEBUG - run - Starting analysis...
2023-01-19 09:43:18,126 - DEBUG - skopeo - Attempting to pull image "docker-daemon:node:18.12.1"
2023-01-19 09:43:18,126 - DEBUG - rootfs - Running command: skopeo copy docker-daemon:node:18.12.1 dir:/root/.tern/temp
2023-01-19 09:43:41,988 - DEBUG - rootfs - Running command: tar -tf /root/.tern/temp/58861e9c7a9b85fcb52553413d8725a7ae8d215349b6325cfdb9d0c38b7f5583
2023-01-19 09:43:42,206 - DEBUG - rootfs - Running command: tar -x -f /root/.tern/temp/58861e9c7a9b85fcb52553413d8725a7ae8d215349b6325cfdb9d0c38b7f5583 -C /root/.tern/temp/1/contents
...
2023-01-19 09:47:44,695 - DEBUG - rootfs - Running command: cp -r /root/.tern/temp/2/contents/app /root/.tern/temp/2/contents/bin /root/.tern/temp/2/contents/etc /root/.tern/temp/2/contents/lib /root/.tern/temp/2/contents/opt /root/.tern/temp/2/contents/sbin /root/.tern/temp/2/contents/tmp /root/.tern/temp/2/contents/usr /root/.tern/temp/2/contents/var /root/.tern/temp/mergedir
2023-01-19 09:48:13,704 - DEBUG - generator - Creating a detailed report of components in image...
2023-01-19 09:48:14,319 - DEBUG - rootfs - Running command: rm -rf /root/.tern/temp/mergedir
2023-01-19 09:48:16,265 - DEBUG - rootfs - Running command: rm -rf /root/.tern/temp/workdir
2023-01-19 09:48:16,267 - DEBUG - rootfs - Running command: rm -rf /root/.tern/temp/1/contents
2023-01-19 09:48:16,568 - DEBUG - rootfs - Running command: rm -rf /root/.tern/temp/2/contents
2023-01-19 09:48:18,193 - DEBUG - prep - Tearing down...
2023-01-19 09:48:29,127 - DEBUG - __main__ - Finished

[rnjudge]

Thanks @amallayev, I am going to close this issue.

[kiranravindran90]

@rnjudge Thank you for the update. I shall try it soon & let know. But seeing previous reply, seems it resolved. 👍