WSS agent is not able to scan docker images .tar files with gzip layers

[pc-jedi]

If I save the docker images with docker save everything is working properly, but if I use the go-containerregistry format for creating the .tar file, then WSS agent fails with the following error:

[DEBUG] [2022-01-12 11:13:47,206 +0100] - Proxy host not provided
[DEBUG] [2022-01-12 11:13:47,214 +0100] - Base url:'https://saas.whitesourcesoftware.com' 
[INFO] [2022-01-12 11:13:47,343 +0100[ - UnifiedAgent version (pluginVersion) : 21.12.2
[INFO] [2022-01-12 11:13:47,349 +0100[ - 
...
------------------------------------------------------------------------
-------------------- Start: Docker Resolver Scan -----------------------
------------------------------------------------------------------------
[DEBUG] [2022-01-12 11:13:47,359 +0100[ - Scanning /Users/xxxx/Projects/test/image.tar
[DEBUG] [2022-01-12 11:13:47,374 +0100[ - DockerUtils - filterTarFiles - START - 1
[INFO] [2022-01-12 11:13:47,374 +0100[ - Filtering docker images list by includes and excludes lists
[DEBUG] [2022-01-12 11:13:47,374 +0100[ - DockerUtils - filterTarFiles - END - 1
[DEBUG] [2022-01-12 11:13:47,394 +0100[ - TarDiscoverer - buildEntitiesFromFiles - START - 1 files
[INFO] [2022-01-12 11:13:47,394 +0100[ - file 1: /Users/xxxx/Projects/test/image.tar
[DEBUG] [2022-01-12 11:13:47,395 +0100[ - TarDiscoverer - isOCILayOut - START - scan image.tar
[DEBUG] [2022-01-12 11:13:47,406 +0100[ - TarDiscoverer - isOCILayOut - blobs:false index:false ocilayout:false
[DEBUG] [2022-01-12 11:13:47,407 +0100[ - TarDiscoverer - isOCILayOut - END - result:false
[DEBUG] [2022-01-12 11:13:47,409 +0100[ - TarDiscoverer - buildEntitiesFromFiles - END - 1 entries
[INFO] [2022-01-12 11:13:47,409 +0100[ - Handle 1 docker images
[INFO] [2022-01-12 11:13:47,410 +0100[ - Image 1 of 1
[INFO] [2022-01-12 11:13:47,410 +0100[ - 
------------------------------------------------------------------------
-------------------- Start: Docker Tar File ----------------------------
------------------------------------------------------------------------
[INFO] [2022-01-12 11:13:47,781 +0100[ - Extracting file /private/var/folders/hr/3hk8cn2x4t568j2423cpc4rm0000gn/T/ws-ua_20220112111347_OLUZPI/Docker_NUPKVM/20220112111347/image.tar - Size 489350656 Bytes (466 MBs)- Free Space 777072738304 Bytes (741074 MBs)
[DEBUG] [2022-01-12 11:13:47,782 +0100[ - AbstractLayerScanner - scan - START - image.tar - /var/folders/hr/3hk8cn2x4t568j2423cpc4rm0000gn/T/ws-ua_20220112111347_OLUZPI/Docker_NUPKVM/20220112111347/image.tar
[DEBUG] [2022-01-12 11:13:47,782 +0100[ - DockerEntityExtraction - extractTarFile - START - /var/folders/hr/3hk8cn2x4t568j2423cpc4rm0000gn/T/ws-ua_20220112111347_OLUZPI/Docker_NUPKVM/20220112111347/image.tar
[DEBUG] [2022-01-12 11:13:48,849 +0100[ - DockerEntityExtraction - extractTarFile - END - true
[DEBUG] [2022-01-12 11:13:48,849 +0100[ - DockerImageScanner - scanLayers - START - Scanning image 'image.tar'
[DEBUG] [2022-01-12 11:13:48,849 +0100[ - DockerUtils - getImageManifest - START - /var/folders/hr/3hk8cn2x4t568j2423cpc4rm0000gn/T/ws-ua_20220112111347_OLUZPI/Docker_NUPKVM/20220112111347
[DEBUG] [2022-01-12 11:13:48,896 +0100[ - DockerUtils - getImageManifest - END - true
[DEBUG] [2022-01-12 11:13:48,896 +0100[ - get image layers file names from image manifest file in '/var/folders/hr/3hk8cn2x4t568j2423cpc4rm0000gn/T/ws-ua_20220112111347_OLUZPI/Docker_NUPKVM/20220112111347'
[DEBUG] [2022-01-12 11:13:48,896 +0100[ - DockerUtils - getDockerImageInfo - START - /var/folders/hr/3hk8cn2x4t568j2423cpc4rm0000gn/T/ws-ua_20220112111347_OLUZPI/Docker_NUPKVM/20220112111347
[WARN] [2022-01-12 11:13:48,897 +0100[ - failed to parse docker config file /var/folders/hr/3hk8cn2x4t568j2423cpc4rm0000gn/T/ws-ua_20220112111347_OLUZPI/Docker_NUPKVM/20220112111347/sha256:7ae0d930f715a751ad98e0c457b2989aceb1670d218f7239bce05a55fe82262f
[DEBUG] [2022-01-12 11:13:48,897 +0100[ - DockerUtils - getDockerImageInfo - END - false
[ERROR] [2022-01-12 11:13:48,897 +0100[ - failed to parse docker image metadata
[DEBUG] [2022-01-12 11:13:48,897 +0100[ - DockerImageScanner - scanLayers - END - failed to parse docker image metadata
[DEBUG] [2022-01-12 11:13:48,897 +0100[ - AbstractLayerScanner - identifyBaseImage - START - image.tar
[DEBUG] [2022-01-12 11:13:48,897 +0100[ - AbstractLayerScanner - identifyBaseImage - END - 0
[DEBUG] [2022-01-12 11:13:48,897 +0100[ - AbstractLayerScanner - getDependencyInfosFromLayers - START - /var/folders/hr/3hk8cn2x4t568j2423cpc4rm0000gn/T/ws-ua_20220112111347_OLUZPI/Docker_NUPKVM/20220112111347 6
[INFO] [2022-01-12 11:13:48,903 +0100[ - AbstractLayerScanner - scanLayersInManifest - START
[DEBUG] [2022-01-12 11:13:48,903 +0100[ - DockerEntityExtraction - extractTarFile - START - /var/folders/hr/3hk8cn2x4t568j2423cpc4rm0000gn/T/ws-ua_20220112111347_OLUZPI/Docker_NUPKVM/20220112111347/8b8a142162d22658bdf0283afcd00a9dd216c6637943ffe5f2ba53c4e3da0bd9.tar.gz
[DEBUG] [2022-01-12 11:13:48,904 +0100[ - ArchiveExtractor - getArchiveInputStream - /var/folders/hr/3hk8cn2x4t568j2423cpc4rm0000gn/T/ws-ua_20220112111347_OLUZPI/Docker_NUPKVM/20220112111347/8b8a142162d22658bdf0283afcd00a9dd216c6637943ffe5f2ba53c4e3da0bd9.tar.gz null
[WARN] [2022-01-12 11:13:48,904 +0100[ - Error extracting file 8b8a142162d22658bdf0283afcd00a9dd216c6637943ffe5f2ba53c4e3da0bd9.tar.gz: Cannot invoke "String.hashCode()" because "<local4>" is null
[DEBUG] [2022-01-12 11:13:48,904 +0100[ - DockerEntityExtraction - extractTarFile - END - false
[INFO] [2022-01-12 11:13:48,904 +0100[ - Did not extract file 8b8a142162d22658bdf0283afcd00a9dd216c6637943ffe5f2ba53c4e3da0bd9.tar.gz
[DEBUG] [2022-01-12 11:13:48,904 +0100[ - AbstractLayerScanner - scanLayersInManifest - failed to extract file 8b8a142162d22658bdf0283afcd00a9dd216c6637943ffe5f2ba53c4e3da0bd9.tar.gz
[DEBUG] [2022-01-12 11:13:48,904 +0100[ - DockerEntityExtraction - extractTarFile - START - /var/folders/hr/3hk8cn2x4t568j2423cpc4rm0000gn/T/ws-ua_20220112111347_OLUZPI/Docker_NUPKVM/20220112111347/a9bd5901c37d1c230822f6597ed14ead62137e086e71600be02ddea7803634e7.tar.gz
[DEBUG] [2022-01-12 11:13:48,905 +0100[ - ArchiveExtractor - getArchiveInputStream - /var/folders/hr/3hk8cn2x4t568j2423cpc4rm0000gn/T/ws-ua_20220112111347_OLUZPI/Docker_NUPKVM/20220112111347/a9bd5901c37d1c230822f6597ed14ead62137e086e71600be02ddea7803634e7.tar.gz null
[WARN] [2022-01-12 11:13:48,905 +0100[ - Error extracting file a9bd5901c37d1c230822f6597ed14ead62137e086e71600be02ddea7803634e7.tar.gz: Cannot invoke "String.hashCode()" because "<local4>" is null
[DEBUG] [2022-01-12 11:13:48,905 +0100[ - DockerEntityExtraction - extractTarFile - END - false
[INFO] [2022-01-12 11:13:48,905 +0100[ - Did not extract file a9bd5901c37d1c230822f6597ed14ead62137e086e71600be02ddea7803634e7.tar.gz
[DEBUG] [2022-01-12 11:13:48,905 +0100[ - AbstractLayerScanner - scanLayersInManifest - failed to extract file a9bd5901c37d1c230822f6597ed14ead62137e086e71600be02ddea7803634e7.tar.gz
[DEBUG] [2022-01-12 11:13:48,905 +0100[ - DockerEntityExtraction - extractTarFile - START - /var/folders/hr/3hk8cn2x4t568j2423cpc4rm0000gn/T/ws-ua_20220112111347_OLUZPI/Docker_NUPKVM/20220112111347/bbb11f9128797ca9d6b7d6bd9538a3753d52e6db9957f459c3ccf4f752b2ba08.tar.gz
[DEBUG] [2022-01-12 11:13:48,906 +0100[ - ArchiveExtractor - getArchiveInputStream - /var/folders/hr/3hk8cn2x4t568j2423cpc4rm0000gn/T/ws-ua_20220112111347_OLUZPI/Docker_NUPKVM/20220112111347/bbb11f9128797ca9d6b7d6bd9538a3753d52e6db9957f459c3ccf4f752b2ba08.tar.gz null
[WARN] [2022-01-12 11:13:48,906 +0100[ - Error extracting file bbb11f9128797ca9d6b7d6bd9538a3753d52e6db9957f459c3ccf4f752b2ba08.tar.gz: Cannot invoke "String.hashCode()" because "<local4>" is null
[DEBUG] [2022-01-12 11:13:48,906 +0100[ - DockerEntityExtraction - extractTarFile - END - false
[INFO] [2022-01-12 11:13:48,906 +0100[ - Did not extract file bbb11f9128797ca9d6b7d6bd9538a3753d52e6db9957f459c3ccf4f752b2ba08.tar.gz
[DEBUG] [2022-01-12 11:13:48,906 +0100[ - AbstractLayerScanner - scanLayersInManifest - failed to extract file bbb11f9128797ca9d6b7d6bd9538a3753d52e6db9957f459c3ccf4f752b2ba08.tar.gz
[DEBUG] [2022-01-12 11:13:48,906 +0100[ - DockerEntityExtraction - extractTarFile - START - /var/folders/hr/3hk8cn2x4t568j2423cpc4rm0000gn/T/ws-ua_20220112111347_OLUZPI/Docker_NUPKVM/20220112111347/ffe985fdf6a85e0e6682bacb597bf3aed6a2cab08403ad556f75c7cafdce874c.tar.gz
[DEBUG] [2022-01-12 11:13:48,923 +0100[ - ArchiveExtractor - getArchiveInputStream - /var/folders/hr/3hk8cn2x4t568j2423cpc4rm0000gn/T/ws-ua_20220112111347_OLUZPI/Docker_NUPKVM/20220112111347/ffe985fdf6a85e0e6682bacb597bf3aed6a2cab08403ad556f75c7cafdce874c.tar.gz null
[WARN] [2022-01-12 11:13:48,923 +0100[ - Error extracting file ffe985fdf6a85e0e6682bacb597bf3aed6a2cab08403ad556f75c7cafdce874c.tar.gz: Cannot invoke "String.hashCode()" because "<local4>" is null
[DEBUG] [2022-01-12 11:13:48,923 +0100[ - DockerEntityExtraction - extractTarFile - END - false
[INFO] [2022-01-12 11:13:48,924 +0100[ - Did not extract file ffe985fdf6a85e0e6682bacb597bf3aed6a2cab08403ad556f75c7cafdce874c.tar.gz
[DEBUG] [2022-01-12 11:13:48,924 +0100[ - AbstractLayerScanner - scanLayersInManifest - failed to extract file ffe985fdf6a85e0e6682bacb597bf3aed6a2cab08403ad556f75c7cafdce874c.tar.gz
[DEBUG] [2022-01-12 11:13:48,924 +0100[ - DockerEntityExtraction - extractTarFile - START - /var/folders/hr/3hk8cn2x4t568j2423cpc4rm0000gn/T/ws-ua_20220112111347_OLUZPI/Docker_NUPKVM/20220112111347/d4da8abc34e9a2cecf0517d0c31715cf9d26101ec76ab8ebdb13dbae2e5a02ad.tar.gz
[DEBUG] [2022-01-12 11:13:48,925 +0100[ - ArchiveExtractor - getArchiveInputStream - /var/folders/hr/3hk8cn2x4t568j2423cpc4rm0000gn/T/ws-ua_20220112111347_OLUZPI/Docker_NUPKVM/20220112111347/d4da8abc34e9a2cecf0517d0c31715cf9d26101ec76ab8ebdb13dbae2e5a02ad.tar.gz null
[WARN] [2022-01-12 11:13:48,925 +0100[ - Error extracting file d4da8abc34e9a2cecf0517d0c31715cf9d26101ec76ab8ebdb13dbae2e5a02ad.tar.gz: Cannot invoke "String.hashCode()" because "<local4>" is null
[DEBUG] [2022-01-12 11:13:48,925 +0100[ - DockerEntityExtraction - extractTarFile - END - false
[INFO] [2022-01-12 11:13:48,925 +0100[ - Did not extract file d4da8abc34e9a2cecf0517d0c31715cf9d26101ec76ab8ebdb13dbae2e5a02ad.tar.gz
[DEBUG] [2022-01-12 11:13:48,925 +0100[ - AbstractLayerScanner - scanLayersInManifest - failed to extract file d4da8abc34e9a2cecf0517d0c31715cf9d26101ec76ab8ebdb13dbae2e5a02ad.tar.gz
[DEBUG] [2022-01-12 11:13:48,925 +0100[ - DockerEntityExtraction - extractTarFile - START - /var/folders/hr/3hk8cn2x4t568j2423cpc4rm0000gn/T/ws-ua_20220112111347_OLUZPI/Docker_NUPKVM/20220112111347/b39dad574c8e281b2b0d988cccbf1efb110a42c7695d2fdc8a721130b7f88216.tar.gz
[DEBUG] [2022-01-12 11:13:48,926 +0100[ - ArchiveExtractor - getArchiveInputStream - /var/folders/hr/3hk8cn2x4t568j2423cpc4rm0000gn/T/ws-ua_20220112111347_OLUZPI/Docker_NUPKVM/20220112111347/b39dad574c8e281b2b0d988cccbf1efb110a42c7695d2fdc8a721130b7f88216.tar.gz null
[WARN] [2022-01-12 11:13:48,926 +0100[ - Error extracting file b39dad574c8e281b2b0d988cccbf1efb110a42c7695d2fdc8a721130b7f88216.tar.gz: Cannot invoke "String.hashCode()" because "<local4>" is null
[DEBUG] [2022-01-12 11:13:48,926 +0100[ - DockerEntityExtraction - extractTarFile - END - false
[INFO] [2022-01-12 11:13:48,926 +0100[ - Did not extract file b39dad574c8e281b2b0d988cccbf1efb110a42c7695d2fdc8a721130b7f88216.tar.gz
[DEBUG] [2022-01-12 11:13:48,926 +0100[ - AbstractLayerScanner - scanLayersInManifest - failed to extract file b39dad574c8e281b2b0d988cccbf1efb110a42c7695d2fdc8a721130b7f88216.tar.gz
[INFO] [2022-01-12 11:13:48,926 +0100[ - AbstractLayerScanner - scanLayersInManifest - END
[DEBUG] [2022-01-12 11:13:48,926 +0100[ - DockerImageSystemPackagesManager - cleanResultFrom - No deleted files were detected
[DEBUG] [2022-01-12 11:13:48,927 +0100[ - AbstractLayerScanner - getDependencyInfosFromLayers - END
[DEBUG] [2022-01-12 11:13:48,928 +0100[ - DockerImageScanner - scanLayers - END - Found 0 system packages
[DEBUG] [2022-01-12 11:13:49,036 +0100[ - AbstractLayerScanner - scan - END - 0 dependencies found in /var/folders/hr/3hk8cn2x4t568j2423cpc4rm0000gn/T/ws-ua_20220112111347_OLUZPI/Docker_NUPKVM/20220112111347/image.tar
[INFO] [2022-01-12 11:13:49,037 +0100[ - 
------------------------------------------------------------------------
-------------------- End: Docker Tar File ------------------------------
------------------------------------------------------------------------
[INFO] [2022-01-12 11:13:49,039 +0100[ - 
------------------------------------------------------------------------
-------------------- End: Docker Resolver Scan -------------------------
------------------------------------------------------------------------

To me it looks like the difference is that the layers in the "legacy' format are only .tar files, while in the new format the layers are compressed as gzip so .tar.gz files. docker load does work with the ".tar.gz" layers.

[nabeelsaabna]

Hi @pc-jedi, Thanks for raising this issue. Could you please open a support ticket with all the relevant information ?

Thanks!

[pc-jedi]

@nabeelsaabna How can I do this?

[LenaKleyner]

@pc-jedi please open a Support Case within our Support portal

[pc-jedi]

I did so. The answer was that "tar.gz layers scan is not supported", which is sad, because that is, except from docker, the most common layer for container images.

But according to the release notes of 2.22.2 I see the following:

Resolved Issues (UA) Unable to extract .tar.gz layers when scanning a docker tar file

[LenaKleyner]

Hi @pc-jedi there was indeed an issue with scanning tar.gz layers which was resolved in release 22.2.2. Does your issue persist?

[pc-jedi]

Yes, that resolved my issue. Thanks for that.