optiopay / klar

Integration of Clair and Docker Registry
MIT License
507 stars 140 forks source link

Push layer n failed: Can't even read an error message: invalid character 'N' looking for beginning of value #55

Closed grebois closed 6 years ago

grebois commented 6 years ago

I been trying to scan local images and I get;

$ docker images | grep debian
debian                                      wheezy              0ec46eb38976        4 days ago          85.1MB

$ CLAIR_ADDR=192.168.99.100 CLAIR_OUTPUT=High CLAIR_THRESHOLD=10 klar debian         
Analysing 1 layers
Push layer 0 failed: Can't even read an error message: invalid character 'N' looking for beginning of value
Analyse image https://registry-1.docker.io/v2/library/debian:latest failed: Analyze error 404: Not Found

Found 0 vulnerabilities

$ 

Same for remote images:

$ CLAIR_ADDR=192.168.99.100 CLAIR_OUTPUT=High CLAIR_THRESHOLD=10 klar postgres:latest
Analysing 12 layers
Push layer 11 failed: Can't even read an error message: invalid character 'N' looking for beginning of value
Push layer 10 failed: Can't even read an error message: invalid character 'N' looking for beginning of value
Push layer 9 failed: Can't even read an error message: invalid character 'N' looking for beginning of value
Push layer 8 failed: Can't even read an error message: invalid character 'N' looking for beginning of value
Push layer 7 failed: Can't even read an error message: invalid character 'N' looking for beginning of value
Push layer 6 failed: Can't even read an error message: invalid character 'N' looking for beginning of value
Push layer 5 failed: Can't even read an error message: invalid character 'N' looking for beginning of value
Push layer 4 failed: Can't even read an error message: invalid character 'N' looking for beginning of value
Push layer 3 failed: Can't even read an error message: invalid character 'N' looking for beginning of value
Push layer 2 failed: Can't even read an error message: invalid character 'N' looking for beginning of value
Push layer 1 failed: Can't even read an error message: invalid character 'N' looking for beginning of value
Push layer 0 failed: Can't even read an error message: invalid character 'N' looking for beginning of value
Analyse image https://registry-1.docker.io/v2/library/postgres:latest failed: Analyze error 404: Not Found

Found 0 vulnerabilities

$ docker pull postgres:latest
latest: Pulling from library/postgres
aa18ad1a0d33: Pull complete 
986b6272a22e: Pull complete 
a23afadd4a20: Pull complete 
b6de223fb1ca: Pull complete 
9656638a0c77: Pull complete 
46a1c4f90191: Pull complete 
ac765cd48f50: Pull complete 
c48f2447ba76: Pull complete 
affdbd879518: Pull complete 
d9cad1b8a255: Pull complete 
72d207c99115: Pull complete 
0fea6738d3c5: Pull complete 
Digest: sha256:2f8080b9910a8b4f38ff5a55a82e77cb43d88bdbb16d723c71d18493590832e9
Status: Downloaded newer image for postgres:latest

$ CLAIR_ADDR=192.168.99.100 CLAIR_OUTPUT=High CLAIR_THRESHOLD=10 klar postgres:latest
Analysing 12 layers
Push layer 11 failed: Can't even read an error message: invalid character 'N' looking for beginning of value
Push layer 10 failed: Can't even read an error message: invalid character 'N' looking for beginning of value
Push layer 9 failed: Can't even read an error message: invalid character 'N' looking for beginning of value
Push layer 8 failed: Can't even read an error message: invalid character 'N' looking for beginning of value
Push layer 7 failed: Can't even read an error message: invalid character 'N' looking for beginning of value
Push layer 6 failed: Can't even read an error message: invalid character 'N' looking for beginning of value
Push layer 5 failed: Can't even read an error message: invalid character 'N' looking for beginning of value
Push layer 4 failed: Can't even read an error message: invalid character 'N' looking for beginning of value
Push layer 3 failed: Can't even read an error message: invalid character 'N' looking for beginning of value
Push layer 2 failed: Can't even read an error message: invalid character 'N' looking for beginning of value
Push layer 1 failed: Can't even read an error message: invalid character 'N' looking for beginning of value
Push layer 0 failed: Can't even read an error message: invalid character 'N' looking for beginning of value
Analyse image https://registry-1.docker.io/v2/library/postgres:latest failed: Analyze error 404: Not Found

Found 0 vulnerabilities
$ 

Is this a bug or am I doing something wrong?

grebois commented 6 years ago

@hashmap is there any way to get a more verbose output? now I'm just getting;

$ CLAIR_ADDR=192.168.99.100 CLAIR_OUTPUT=High CLAIR_THRESHOLD=10 klar postgres:latest
Can't pull fsLayers
hashmap commented 6 years ago

@grebois thanks for the report, could you build a binary from the source? Use this branch https://github.com/optiopay/klar/tree/remove-distribution-list or let me know - I can provide you with the binary for your OS.

grebois commented 6 years ago

@hashmap sure, this is the output;

$ git clone git@github.com:optiopay/klar.git
Cloning into 'klar'...
remote: Counting objects: 214, done.
remote: Compressing objects: 100% (9/9), done.
remote: Total 214 (delta 3), reused 7 (delta 2), pack-reused 203
Receiving objects: 100% (214/214), 291.97 KiB | 73.00 KiB/s, done.
Resolving deltas: 100% (98/98), done.

$ git checkout remove-distribution-list 
fatal: Not a git repository (or any of the parent directories): .git

$ cd klar/
$ git checkout
Your branch is up-to-date with 'origin/master'.

$ git checkout remove-distribution-list 
Branch remove-distribution-list set up to track remote branch remove-distribution-list from origin.
Switched to a new branch 'remove-distribution-list'

$ go build .
$ ls
Dockerfile  LICENSE     README.md   assets      clair       docker      klar        main.go

$ ./klar 
Image name must be provided

$ CLAIR_ADDR=192.168.99.100 CLAIR_OUTPUT=High CLAIR_THRESHOLD=10 ./klar postgres:latest
Can't pull fsLayers

$ 
xueshanf commented 6 years ago

@hashmap same here. Both v1.4.1 and the build from remove-distribution-list branch return Can't pull fsLayers. Images can be scanned before with v1.4.1 now getting this error. Older build (docker 1.9) postgres:9.5.2still can be scanned, but postgres:latest cannot.

hashmap commented 6 years ago

Please try v1.5 RC1 https://github.com/optiopay/klar/releases/tag/1.5-RC1

xueshanf commented 6 years ago

@hashmap 1.5-rc1 works! I built docker image and it is available here: https://hub.docker.com/r/xueshanf/klar/.

hashmap commented 6 years ago

If you still have any issues please try https://github.com/optiopay/klar/releases/tag/v1.5-RC2 it's basically the same version but with simple tracing support, specify env var to enable it: KLAR_TRACE=true

MansM commented 6 years ago

I am still getting this issue, but when googling around I noticed clairctl has the same issue due a change in the docker hub output ( see https://github.com/jgsqware/clairctl/issues/93#issuecomment-333490590 ). Maybe related?

hashmap commented 6 years ago

@MansM could you try to run https://github.com/optiopay/klar/releases/tag/v1.5-RC2 with KLAR_TRACE=true env var and share output here?

MansM commented 6 years ago
KLAR_TRACE=true CLAIR_ADDR=http://clair.example.com:80 ./klar-1.5-RC2-osx-amd64 centos:7
----> HTTP REQUEST:
GET /v2/library/centos/manifests/7 HTTP/1.1
Host: registry-1.docker.io
Accept: application/vnd.docker.distribution.manifest.v2+json

<---- HTTP RESPONSE:
HTTP/1.1 401 Unauthorized
Content-Length: 157
Content-Type: application/json; charset=utf-8
Date: Mon, 09 Oct 2017 08:39:14 GMT
Docker-Distribution-Api-Version: registry/2.0
Strict-Transport-Security: max-age=31536000
Www-Authenticate: Bearer realm="https://auth.docker.io/token",service="registry.docker.io",scope="repository:library/centos:pull"

{"errors":[{"code":"UNAUTHORIZED","message":"authentication required","detail":[{"Type":"repository","Class":"","Name":"library/centos","Action":"pull"}]}]}

----> HTTP REQUEST:
GET /v2/library/centos/manifests/7 HTTP/1.1
Host: registry-1.docker.io
Accept: application/vnd.docker.distribution.manifest.v2+json
Authorization: Bearer eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCIsIng1YyI6WyJNSUlDTHpDQ0FkU2dBd0lCQWdJQkFEQUtCZ2dxaGtqT1BRUURBakJHTVVRd1FnWURWUVFERXp0Uk5Gb3pPa2RYTjBrNldGUlFSRHBJVFRSUk9rOVVWRmc2TmtGRlF6cFNUVE5ET2tGU01rTTZUMFkzTnpwQ1ZrVkJPa2xHUlVrNlExazFTekFlRncweE56QTFNREl5TWpBME5UZGFGdzB4T0RBMU1ESXlNakEwTlRkYU1FWXhSREJDQmdOVkJBTVRPMDFPTms0NlJraFVWenBKV0VWSE9rOUpOMUU2UVRWWFJqcFpSRVUwT2pkRE4wNDZSMWRKVVRvMVZ6STNPa2hPTlVvNlZVNURRVG95U0UxQ01Ga3dFd1lIS29aSXpqMENBUVlJS29aSXpqMERBUWNEUWdBRU5KRklhQ1hHNWYxSk9BZnZSaTJDU081K1Q5RVpKd2doai9SUXgzNW9Uc3Q4RnhXY0dRc3ZOMG5sdW5DVVdIbENxN2I4NFJRTXV0WUVIUnY4MVhweTU2T0JzakNCcnpBT0JnTlZIUThCQWY4RUJBTUNCNEF3RHdZRFZSMGxCQWd3QmdZRVZSMGxBREJFQmdOVkhRNEVQUVE3VFU0MlRqcEdTRlJYT2tsWVJVYzZUMGszVVRwQk5WZEdPbGxFUlRRNk4wTTNUanBIVjBsUk9qVlhNamM2U0U0MVNqcFZUa05CT2pKSVRVSXdSZ1lEVlIwakJEOHdQWUE3VVRSYU16cEhWemRKT2xoVVVFUTZTRTAwVVRwUFZGUllPalpCUlVNNlVrMHpRenBCVWpKRE9rOUdOemM2UWxaRlFUcEpSa1ZKT2tOWk5Vc3dDZ1lJS29aSXpqMEVBd0lEU1FBd1JnSWhBSTJVUlpMQVRTM3R4bjNpNTY0SXVQSFEwQU1Mb1g5cTZCMmdnN01KSHJuTkFpRUE0Q3lzbmtENHhjQm42amdobVdnQzczQjdGVkszenFnOTV4ZjNRK2xGVHlrPSJdfQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImxpYnJhcnkvY2VudG9zIiwiYWN0aW9ucyI6WyJwdWxsIl19XSwiYXVkIjoicmVnaXN0cnkuZG9ja2VyLmlvIiwiZXhwIjoxNTA3NTM4NjU0LCJpYXQiOjE1MDc1MzgzNTQsImlzcyI6ImF1dGguZG9ja2VyLmlvIiwianRpIjoiSzRmY0I5U3pLbEFVQTZmMjhrVHoiLCJuYmYiOjE1MDc1MzgwNTQsInN1YiI6IiJ9.rpu8iKViBGzyDoXR227K4ramMkJY2K8Dqgytg4szzYNW620xSueike_h2guzJdLMcpsSDGz8yNfQq7AkaPqn-w

<---- HTTP RESPONSE:
HTTP/1.1 200 OK
Content-Length: 529
Content-Type: application/vnd.docker.distribution.manifest.v2+json
Date: Mon, 09 Oct 2017 08:39:14 GMT
Docker-Content-Digest: sha256:eba772bac22c86d7d6e72421b4700c3f894ab6e35475a34014ff8de74c10872e
Docker-Distribution-Api-Version: registry/2.0
Etag: "sha256:eba772bac22c86d7d6e72421b4700c3f894ab6e35475a34014ff8de74c10872e"
Strict-Transport-Security: max-age=31536000

{
   "schemaVersion": 2,
   "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
   "config": {
      "mediaType": "application/vnd.docker.container.image.v1+json",
      "size": 1863,
      "digest": "sha256:196e0ce0c9fbb31da595b893dd39bc9fd4aa78a474bbdc21459a3ebe855b7768"
   },
   "layers": [
      {
         "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
         "size": 73386947,
         "digest": "sha256:d9aaf4d82f249dc101a6638ff5177fe926cdebfa6c42d874dfa5029533da0e72"
      }
   ]
}
Analysing 1 layers
----> HTTP REQUEST:
POST /v1/layers HTTP/1.1
Host: clair.example.com:80
Content-Type: application/json

{"Layer":{"Name":"196e0ce0c9fbb31da595b893dd39bc9fd4aa78a474bbdc21459a3ebe855b7768d9aaf4d82f249dc101a6638ff5177fe926cdebfa6c42d874dfa5029533da0e72","Path":"https://registry-1.docker.io/v2/library/centos/blobs/sha256:d9aaf4d82f249dc101a6638ff5177fe926cdebfa6c42d874dfa5029533da0e72","ParentName":"","Format":"Docker","Features":null,"Headers":{"Authorization":"Bearer eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCIsIng1YyI6WyJNSUlDTHpDQ0FkU2dBd0lCQWdJQkFEQUtCZ2dxaGtqT1BRUURBakJHTVVRd1FnWURWUVFERXp0Uk5Gb3pPa2RYTjBrNldGUlFSRHBJVFRSUk9rOVVWRmc2TmtGRlF6cFNUVE5ET2tGU01rTTZUMFkzTnpwQ1ZrVkJPa2xHUlVrNlExazFTekFlRncweE56QTFNREl5TWpBME5UZGFGdzB4T0RBMU1ESXlNakEwTlRkYU1FWXhSREJDQmdOVkJBTVRPMDFPTms0NlJraFVWenBKV0VWSE9rOUpOMUU2UVRWWFJqcFpSRVUwT2pkRE4wNDZSMWRKVVRvMVZ6STNPa2hPTlVvNlZVNURRVG95U0UxQ01Ga3dFd1lIS29aSXpqMENBUVlJS29aSXpqMERBUWNEUWdBRU5KRklhQ1hHNWYxSk9BZnZSaTJDU081K1Q5RVpKd2doai9SUXgzNW9Uc3Q4RnhXY0dRc3ZOMG5sdW5DVVdIbENxN2I4NFJRTXV0WUVIUnY4MVhweTU2T0JzakNCcnpBT0JnTlZIUThCQWY4RUJBTUNCNEF3RHdZRFZSMGxCQWd3QmdZRVZSMGxBREJFQmdOVkhRNEVQUVE3VFU0MlRqcEdTRlJYT2tsWVJVYzZUMGszVVRwQk5WZEdPbGxFUlRRNk4wTTNUanBIVjBsUk9qVlhNamM2U0U0MVNqcFZUa05CT2pKSVRVSXdSZ1lEVlIwakJEOHdQWUE3VVRSYU16cEhWemRKT2xoVVVFUTZTRTAwVVRwUFZGUllPalpCUlVNNlVrMHpRenBCVWpKRE9rOUdOemM2UWxaRlFUcEpSa1ZKT2tOWk5Vc3dDZ1lJS29aSXpqMEVBd0lEU1FBd1JnSWhBSTJVUlpMQVRTM3R4bjNpNTY0SXVQSFEwQU1Mb1g5cTZCMmdnN01KSHJuTkFpRUE0Q3lzbmtENHhjQm42amdobVdnQzczQjdGVkszenFnOTV4ZjNRK2xGVHlrPSJdfQ.eyJhY2Nlc3MiOlt7InR5cGUiOiJyZXBvc2l0b3J5IiwibmFtZSI6ImxpYnJhcnkvY2VudG9zIiwiYWN0aW9ucyI6WyJwdWxsIl19XSwiYXVkIjoicmVnaXN0cnkuZG9ja2VyLmlvIiwiZXhwIjoxNTA3NTM4NjU0LCJpYXQiOjE1MDc1MzgzNTQsImlzcyI6ImF1dGguZG9ja2VyLmlvIiwianRpIjoiSzRmY0I5U3pLbEFVQTZmMjhrVHoiLCJuYmYiOjE1MDc1MzgwNTQsInN1YiI6IiJ9.rpu8iKViBGzyDoXR227K4ramMkJY2K8Dqgytg4szzYNW620xSueike_h2guzJdLMcpsSDGz8yNfQq7AkaPqn-w"}}}
<---- HTTP RESPONSE:
HTTP/1.1 404 Not Found
Content-Length: 10
Connection: keep-alive
Content-Type: text/plain; charset=utf-8
Date: Mon, 09 Oct 2017 08:39:21 GMT
Server: nginx/1.13.5
X-Content-Type-Options: nosniff

Not Found

Push layer 0 failed: Can't even read an error message: invalid character 'N' looking for beginning of value
----> HTTP REQUEST:
GET /v1/layers/196e0ce0c9fbb31da595b893dd39bc9fd4aa78a474bbdc21459a3ebe855b7768d9aaf4d82f249dc101a6638ff5177fe926cdebfa6c42d874dfa5029533da0e72?vulnerabilities HTTP/1.1
Host: clair.example.com:80

<---- HTTP RESPONSE:
HTTP/1.1 404 Not Found
Content-Length: 10
Connection: keep-alive
Content-Type: text/plain; charset=utf-8
Date: Mon, 09 Oct 2017 08:39:21 GMT
Server: nginx/1.13.5
X-Content-Type-Options: nosniff

Not Found

Analyse image https://registry-1.docker.io/v2/library/centos:7 failed: Analyze error 404: Not Found

Found 0 vulnerabilities
MansM commented 6 years ago

ok, found the solution for my issue: dont use quay.io/coreos/clair-git as image... using quay.io/coreos/clair worked fine :-)