search

jfrog / jfrog-cli

JFrog CLI is a client that provides a simple interface that automates access to the JFrog products.
https://www.jfrog.com/confluence/display/CLI/JFrog+CLI
Apache License 2.0
530 stars 227 forks source link

jf docker scan fails on golang/src/archive/tar/testdata/pax-bad-hdr-large.tar.bz2 #1888

Open tomgeorge opened 1 year ago

tomgeorge commented 1 year ago

Describe the bug

Running jf docker scan on a container image based on golang:1.20.1-alpine3.17 fails when trying to index pax-bad-hdr-large.tar.bz2 which is in the container at /usr/local/go/src/archive/tar/testdata/pax-bad-hdr-large.tar.bz2.

Current behavior

Ξ» ~/golang-test/ jf docker scan golang-test
12:25:28 [πŸ”΅Info] Log path: /Users/tom.george/.jfrog/logs/jfrog-cli.2023-03-30.12-25-28.36896.log
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚ ✨ No vulnerabilities were found ✨ β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
12:25:42 [🚨Error] Xray indexer app failed indexing /var/folders/_1/qrh71qqj45xf2jhxwj6scgd40000gp/T/jfrog.cli.temp.-1680197129-1021605106/image.tar with exit status 2: 2023-03-30T17:25:37.464142Z [jfxia] [INFO ] [] [docker_layer_tar:171          ] [main                ] Encountered release info file 'etc/alpine-release'
2023-03-30T17:25:37.464815Z [jfxia] [INFO ] [] [docker_layer_tar:171          ] [main                ] Encountered release info file 'etc/os-release'
2023-03-30T17:25:39.917596Z [jfxia] [INFO ] [] [/usr/local/go/src/sync/once:74] [main                ] SPDX license IDs from licenses.json and exceptions.json were loaded successfully
2023-03-30T17:25:40.969984Z [jfxia] [WARN ] [] [archive_mgr:662               ] [main                ] Archive manifest.json exceeded internal depth limitation, extraction stopped.
2023-03-30T17:25:41.61123Z [jfxia] [WARN ] [] [archive_mgr:281               ] [main                ] failed to extract tar: archive/tar: invalid tar header
2023-03-30T17:25:41.612136Z [jfxia] [WARN ] [] [archive_mgr:281               ] [main                ] failed to extract tar: archive/tar: invalid tar header
2023-03-30T17:25:41.613006Z [jfxia] [WARN ] [] [archive_mgr:281               ] [main                ] failed to extract tar: archive/tar: invalid tar header
2023-03-30T17:25:41.613783Z [jfxia] [WARN ] [] [archive_mgr:281               ] [main                ] failed to extract tar: archive/tar: invalid tar header
2023-03-30T17:25:41.615615Z [jfxia] [WARN ] [] [archive_mgr:281               ] [main                ] failed to extract tar: archive/tar: invalid tar header
2023-03-30T17:25:41.623937Z [jfxia] [ERROR] [] [archive_mgr:227               ] [main                ] Failed to extract layer data of '/var/folders/_1/qrh71qqj45xf2jhxwj6scgd40000gp/T/jfrog.cli.temp.-1680197134-4073455054/cf96fe2d-63af-47ce-7b06-8f145d5f84b0/168019713993117900/sha256__c6bcad44cf36393c281e50b768d575a49a501a64aaba125d7a6d5e6d29690dfa.tar': 'failed to extract file pax-bad-hdr-large.tar.bz2 (root path: /var/folders/_1/qrh71qqj45xf2jhxwj6scgd40000gp/T/jfrog.cli.temp.-1680197134-4073455054/cf96fe2d-63af-47ce-7b06-8f145d5f84b0/168019713651366500/).
 --- at /go/src/jfrog.com/xray/backend/indexer/indexer_core/archive_mgr.go:291 (ArchiveManager.deepArchiveScanWrapper) ---
Caused by: failed to check if 2f7661722f666f6c646572732f5f312f717268373171716a34357866326a6878776a3673636764343030303067702f542f6a66726f672e636c692e74656d702e2d313638303139373133342d343037333435353035342f63663936666532642d363361662d343763652d376230362d3866313435643566383462302f3136383031393731343136313539343930302f7061782d6261642d6864722d6c617267652e7461722e627a32 is a conda package
 --- at /go/src/jfrog.com/xray/backend/indexer/indexer_core/tar.go:74 (TarOpenerFactory.DeepArchiveScan) ---
Caused by: Irrecoverable Error: failed reading tar.bz2 file /var/folders/_1/qrh71qqj45xf2jhxwj6scgd40000gp/T/jfrog.cli.temp.-1680197134-4073455054/cf96fe2d-63af-47ce-7b06-8f145d5f84b0/168019714161594900/pax-bad-hdr-large.tar.bz2
 --- at /go/src/jfrog.com/xray/backend/indexer/indexer_core/conda.go:179 (getIndexJsonContentFromArchiveReader) ---
Caused by: archive/tar: header field too long'
2023-03-30T17:25:41.624321Z [jfxia] [ERROR] [] [docker_tar:99                 ] [main                ] Failed to index temporary file '/var/folders/_1/qrh71qqj45xf2jhxwj6scgd40000gp/T/jfrog.cli.temp.-1680197134-4073455054/cf96fe2d-63af-47ce-7b06-8f145d5f84b0/168019713654542100/9f9a556ab2f4fe3ce505c017c7bdf2ca3249172303697c103a928d94cb4f3bde/sha256__c6bcad44cf36393c281e50b768d575a49a501a64aaba125d7a6d5e6d29690dfa.tar': failed to extract file sha256__c6bcad44cf36393c281e50b768d575a49a501a64aaba125d7a6d5e6d29690dfa.tar (root path: /var/folders/_1/qrh71qqj45xf2jhxwj6scgd40000gp/T/jfrog.cli.temp.-1680197134-4073455054/cf96fe2d-63af-47ce-7b06-8f145d5f84b0/168019713651366500/).
 --- at /go/src/jfrog.com/xray/backend/indexer/indexer_core/archive_mgr.go:291 (ArchiveManager.deepArchiveScanWrapper) ---
Caused by: failed to extract file pax-bad-hdr-large.tar.bz2 (root path: /var/folders/_1/qrh71qqj45xf2jhxwj6scgd40000gp/T/jfrog.cli.temp.-1680197134-4073455054/cf96fe2d-63af-47ce-7b06-8f145d5f84b0/168019713651366500/).
 --- at /go/src/jfrog.com/xray/backend/indexer/indexer_core/archive_mgr.go:291 (ArchiveManager.deepArchiveScanWrapper) ---
Caused by: failed to check if 2f7661722f666f6c646572732f5f312f717268373171716a34357866326a6878776a3673636764343030303067702f542f6a66726f672e636c692e74656d702e2d313638303139373133342d343037333435353035342f63663936666532642d363361662d343763652d376230362d3866313435643566383462302f3136383031393731343136313539343930302f7061782d6261642d6864722d6c617267652e7461722e627a32 is a conda package
 --- at /go/src/jfrog.com/xray/backend/indexer/indexer_core/tar.go:74 (TarOpenerFactory.DeepArchiveScan) ---
Caused by: Irrecoverable Error: failed reading tar.bz2 file /var/folders/_1/qrh71qqj45xf2jhxwj6scgd40000gp/T/jfrog.cli.temp.-1680197134-4073455054/cf96fe2d-63af-47ce-7b06-8f145d5f84b0/168019714161594900/pax-bad-hdr-large.tar.bz2
 --- at /go/src/jfrog.com/xray/backend/indexer/indexer_core/conda.go:179 (getIndexJsonContentFromArchiveReader) ---
Caused by: archive/tar: header field too long
2023-03-30T17:25:41.650578Z [jfxia] [ERROR] [] [proc:250                      ] [main                ] Failed to index '/var/folders/_1/qrh71qqj45xf2jhxwj6scgd40000gp/T/jfrog.cli.temp.-1680197129-1021605106/image.tar': failed to index file '/var/folders/_1/qrh71qqj45xf2jhxwj6scgd40000gp/T/jfrog.cli.temp.-1680197129-1021605106/image.tar'
 --- at /go/src/jfrog.com/xray/backend/indexer/indexer_core/archive_mgr.go:126 (ArchiveManager.IndexStandaloneExtFile) ---
Caused by: failed to extract file /golang-test/latest/manifest.json (root path: /var/folders/_1/qrh71qqj45xf2jhxwj6scgd40000gp/T/jfrog.cli.temp.-1680197134-4073455054/cf96fe2d-63af-47ce-7b06-8f145d5f84b0/168019713651366500/).
 --- at /go/src/jfrog.com/xray/backend/indexer/indexer_core/archive_mgr.go:291 (ArchiveManager.deepArchiveScanWrapper) ---
Caused by: failed to scan docker layer 'sha256__c6bcad44cf36393c281e50b768d575a49a501a64aaba125d7a6d5e6d29690dfa.tar'
 --- at /go/src/jfrog.com/xray/backend/indexer/indexer_core/docker.go:110 (DockerOpener.DeepArchiveScan) ---
Caused by: failed to extract file sha256__c6bcad44cf36393c281e50b768d575a49a501a64aaba125d7a6d5e6d29690dfa.tar (root path: /var/folders/_1/qrh71qqj45xf2jhxwj6scgd40000gp/T/jfrog.cli.temp.-1680197134-4073455054/cf96fe2d-63af-47ce-7b06-8f145d5f84b0/168019713651366500/).
 --- at /go/src/jfrog.com/xray/backend/indexer/indexer_core/archive_mgr.go:291 (ArchiveManager.deepArchiveScanWrapper) ---
Caused by: failed to extract file pax-bad-hdr-large.tar.bz2 (root path: /var/folders/_1/qrh71qqj45xf2jhxwj6scgd40000gp/T/jfrog.cli.temp.-1680197134-4073455054/cf96fe2d-63af-47ce-7b06-8f145d5f84b0/168019713651366500/).
 --- at /go/src/jfrog.com/xray/backend/indexer/indexer_core/archive_mgr.go:291 (ArchiveManager.deepArchiveScanWrapper) ---
Caused by: failed to check if 2f7661722f666f6c646572732f5f312f717268373171716a34357866326a6878776a3673636764343030303067702f542f6a66726f672e636c692e74656d702e2d313638303139373133342d343037333435353035342f63663936666532642d363361662d343763652d376230362d3866313435643566383462302f3136383031393731343136313539343930302f7061782d6261642d6864722d6c617267652e7461722e627a32 is a conda package
 --- at /go/src/jfrog.com/xray/backend/indexer/indexer_core/tar.go:74 (TarOpenerFactory.DeepArchiveScan) ---
Caused by: Irrecoverable Error: failed reading tar.bz2 file /var/folders/_1/qrh71qqj45xf2jhxwj6scgd40000gp/T/jfrog.cli.temp.-1680197134-4073455054/cf96fe2d-63af-47ce-7b06-8f145d5f84b0/168019714161594900/pax-bad-hdr-large.tar.bz2
 --- at /go/src/jfrog.com/xray/backend/indexer/indexer_core/conda.go:179 (getIndexJsonContentFromArchiveReader) ---
Caused by: archive/tar: header field too long

Reproduction steps

FROM golang:1.20.1-alpine3.17

docker build -t golang-test . jf docker scan golang-test

Expected behavior

I expect it to not fail on this piece of test data, or to skip it like many of the invalid tar header errors.

JFrog CLI version

2.35.0

Operating system type and version

Mac/Linux, happens locally and in CI

JFrog Artifactory version

7.55.9

JFrog Xray version

3.69.3

tomgeorge commented 1 year ago

I would also hazard a guess that this affects any golang container. We are instructing teams to not have their runtime containers be FROM golang, but I think xray should know enough to not scan test data.

sverdlov93 commented 1 year ago

Hi @tomgeorge, Thanks for reporting this issue. Our apologies for the inconvenience caused by this. The above issue is already resolved on JFrog Xray side and will be released soon. ( I believe that it will be Xray version 3.71.X and higher) I will update here once it's released.