search

TritonDataCenter / node-docker-registry-client

node.js client for the docker registry
Mozilla Public License 2.0
64 stars 33 forks source link

v2 API's listTags() doesn't support paging #12

Open trentm opened 7 years ago

trentm commented 7 years ago

The Docker Registry v2 API's endpoint to list tags is paginated: https://docs.docker.com/registry/spec/api/#tags-paginated This module's listTags doesn't yet support that.

Currently that is resulting in the "listTags" test in "v2.quayio.test.js" to fail, because the repo used for the test (quay.io/coreos/etcd) now has more than 50 tags (which I guess in quay.io's default page size). Here is some instrumented output from that failing test:

$ ./node_modules/.bin/tape test/v2.quayio.test.js
TAP version 13
# v2 quay.io
#   createClient
ok 1 should be truthy
ok 2 should be equal
#   supportsV2
ok 3 null
ok 4 supportsV2
#   ping
ok 5 should be truthy
ok 6 have a response
ok 7 should be equal
ok 8 should be truthy
ok 9 should be equal
#   listTags
XXX res.headers { server: 'nginx/1.10.0 (Ubuntu)',
  date: 'Mon, 31 Oct 2016 17:56:53 GMT',
  'content-type': 'application/json',
  'content-length': '602',
  connection: 'keep-alive',
  link: '<https://quay.io/v2/coreos/etcd/tags/list?next_page=gAAAAABYF4XlRL00UY6caJ4wGdywTryh8c_MpJfbRTEhQ6e-X4D9MnzHxe0V6UbhxqtGJBqjOvo8_LSmCqOccyvjPs8T4rR0Sw%3D%3D&n=50>; rel="next"',
  'x-frame-options': 'DENY',
  'strict-transport-security': 'max-age=63072000; preload',
  'x-request-received': 1477936612677,
  'x-request-processing-time': 1004 }
ok 10 undefined
ok 11 should be truthy
ok 12 should be equal
not ok 13 no "latest" tag in listTags:{"name":"coreos/etcd","tags":["v0.4.6","v0.5.0_alpha.0","v0.5.0_alpha.1","v0.5.0_alpha.2","v0.5.0_alpha.3","test","v0.5.0_alpha.4","v0.5.0_alpha.5","v2.0.0_rc.1","v2.0.0","v2.0.3","v2.0.4","v2.0.5","v0.4.8","v2.0.6","v2.0.7","v2.0.8","v2.0.9","v2.0.10","v2.1.0-alpha.0","v2.0.11","v2.0.12","v2.1.0-alpha.1","v2.0.13","v2.1.0-rc.0","v2.1.1","v2.2.0-alpha.0","v2.2.0-alpha.1","v2.1.2","v2.2.0-rc.0","v2.1.3","v2.2.0","v2.2.1","v2.3.0-alpha.0","v2.2.2","v2.2.3","v2.2.4","v2.2.5","v2.3.0-alpha.1","v2.3.0","v2.3.1","v2.3.2","v2.3.3","v3.0.0-beta.0","v2.3.4","v2.3.5","v2.3.6","v2.3.7","v3.0.0","v3.0.1"]}
  ---
    operator: ok
    expected: true
    actual:   false
    at: next (/Users/trentm/joy/node-docker-registry-client/node_modules/vasync/lib/vasync.js:201:4)
  ...

Note that Link header: link: '<https://quay.io/v2/coreos/etcd/tags/list?next_page=gAAAAABYF4XlRL00UY6caJ4wGdywTryh8c_MpJfbRTEhQ6e-X4D9MnzHxe0V6UbhxqtGJBqjOvo8_LSmCqOccyvjPs8T4rR0Sw%3D%3D&n=50>; rel="next"',

From the spec link above:

Link:   RFC5988 compliant rel=’next’ with URL to next result set, if available

Example:
Link: <<url>?n=<last n value>&last=<last entry from response>>; rel="next"

So we have an inconsistency between the spec and practice (which has happened before). This might need some poking around to get right.

(Currently AFAICT, Triton doesn't use this listTags endpoint, so I don't have an immediate burning need to implement this.)

Suggestion:

aaronbeall commented 5 years ago

So this has bitten us a few times with our internal tools. We end up with more than 50 tags for an image (happens easily because each feature branch is a new tag) and our internal build tools stop working, everyone is confused until someone who's been around for awhile realizes what's going on and deletes some old tags. It'd be great if this could get addressed. 👍

idelsink commented 4 years ago

I think this issue has been resolved? Like I can get all the tags of a node repository with over 2000+ tags

Update: Nvm it indeed does not support pagination. AWS ECR defaults returns 100 tags and the response header is not followed