search

jfrog / artifactory-scripts

Scripts for Artifactory (Usually, for REST API), community driven.
Apache License 2.0
148 stars 166 forks source link

docker_fetch_markers returns 404 #58

Open Smasherr opened 2 years ago

Smasherr commented 2 years ago

I've tried to run the script but it returned 404 codes. To see more details, I've added set +x to the script and runned it again:

❯ ./docker_marker.sh
+ echo 'Enter your Artifactory URL: '
Enter your Artifactory URL:
+ read Source_ART_URL
https://[MASKED]/artifactory
+ SOURCE_ART=https://[MASKED]/artifactory
+ echo 'Enter your docker remote repository name without the '\''-cache'\'' suffix: '
Enter your docker remote repository name without the '-cache' suffix:
+ read Source_repo_name
registry-gitlab-com
+ echo 'Enter username with Read and Deploy permissions to the above repository in Artifactory: '
Enter username with Read and Deploy permissions to the above repository in Artifactory:
+ read source_username
[MASKED]
+ echo 'Password for Artifactory: '
Password for Artifactory:
+ read -s source_password
+ echo

+ REMOTE_REPO=registry-gitlab-com-cache
++ curl -X POST -sS -u[MASKED] --write-out '%{http_code}' --silent --output /dev/null https://[MASKED]/artifactory/api/search/aql -d 'items.find({"$and": [{"repo" : "registry-gitlab-com-cache"}, {"name" : {"$match" : "*.marker"}}]})' -H 'Content-Type: text/plain'
+ status_code=200
+ [[ 200 -eq 401 ]]
+ [[ 200 -eq 000 ]]
+ [[ 200 -eq 404 ]]
+ [[ 200 -eq 400 ]]
+ [[ 200 -ne 200 ]]
+ curl -X POST -sS -u[MASKED] https://[MASKED]/artifactory/api/search/aql -d 'items.find({"$and": [{"repo" : "registry-gitlab-com-cache"}, {"name" : {"$match" : "*.marker"}}]})' -H 'Content-Type: text/plain'
+ jq -M -r '.results[] | "\(.path)/blobs/\(.name)"' marker_layers.txt
+ sed 's/[“,]//g' marker_paths.txt
+ sed 's|library/||g'
+ sed s/.marker//g
+ sed s/__/:/g
+ awk 'sub("[/][^,;/]+[/]blobs/", "/blobs/", $0)'
+ echo 'Here are the number of marker layers in this repository'
Here are the number of marker layers in this repository
+ echo

+ cat download_markers.txt
+ wc -l
      12
+ echo

+ echo 'Do you want to download these marker layers?(yes/no)'
Do you want to download these marker layers?(yes/no)
+ read input
yes
+ [[ yes =~ [yY](es)* ]]
+ read p
+ prefix=https://[MASKED]/artifactory/api/docker/registry-gitlab-com/v2/gitlab-org/cluster-integration/auto-build-image/blobs/sha256:28d445cfca767cb2a0dbdc152ed5eeed364db2b6353f8dedf8f917562c116b9b
+ read p
+ prefix=https://[MASKED]/artifactory/api/docker/registry-gitlab-com/v2/gitlab-org/cluster-integration/auto-build-image/blobs/sha256:2b7e0e3b754519331113e409b44b7401dda685cba747e0fbb619b907b85e65a6
+ read p
+ curl -sS -u[MASKED] -w 'HTTP/1.1 %{http_code} | %{time_total} seconds | %{size_download} bytes\n' https://[MASKED]/artifactory/api/docker/registry-gitlab-com/v2/gitlab-org/cluster-integration/auto-build-image/blobs/sha256:28d445cfca767cb2a0dbdc152ed5eeed364db2b6353f8dedf8f917562c116b9b -o /dev/null
+ prefix=https://[MASKED]/artifactory/api/docker/registry-gitlab-com/v2/gitlab-org/cluster-integration/auto-build-image/blobs/sha256:2baf61d06a27504c4b8421899469ed523b29ea095f17de8c8a15248304ae4133
+ curl -sS -u[MASKED] -w 'HTTP/1.1 %{http_code} | %{time_total} seconds | %{size_download} bytes\n' https://[MASKED]/artifactory/api/docker/registry-gitlab-com/v2/gitlab-org/cluster-integration/auto-build-image/blobs/sha256:2b7e0e3b754519331113e409b44b7401dda685cba747e0fbb619b907b85e65a6 -o /dev/null
+ read p
+ curl -sS -u[MASKED] -w 'HTTP/1.1 %{http_code} | %{time_total} seconds | %{size_download} bytes\n' https://[MASKED]/artifactory/api/docker/registry-gitlab-com/v2/gitlab-org/cluster-integration/auto-build-image/blobs/sha256:2baf61d06a27504c4b8421899469ed523b29ea095f17de8c8a15248304ae4133 -o /dev/null
+ prefix=https://[MASKED]/artifactory/api/docker/registry-gitlab-com/v2/gitlab-org/cluster-integration/auto-build-image/blobs/sha256:32eb6dfb922720a338efac9871565ad3ca1a3ac69b540f214e1224bc55191c69
+ read p
+ prefix=https://[MASKED]/artifactory/api/docker/registry-gitlab-com/v2/gitlab-org/cluster-integration/auto-build-image/blobs/sha256:493333e9491fba57ff1ea5b8058546fc5a3e124d325f8720c018568f852e385b
+ curl -sS -u[MASKED] -w 'HTTP/1.1 %{http_code} | %{time_total} seconds | %{size_download} bytes\n' https://[MASKED]/artifactory/api/docker/registry-gitlab-com/v2/gitlab-org/cluster-integration/auto-build-image/blobs/sha256:32eb6dfb922720a338efac9871565ad3ca1a3ac69b540f214e1224bc55191c69 -o /dev/null
+ curl -sS -u[MASKED] -w 'HTTP/1.1 %{http_code} | %{time_total} seconds | %{size_download} bytes\n' https://[MASKED]/artifactory/api/docker/registry-gitlab-com/v2/gitlab-org/cluster-integration/auto-build-image/blobs/sha256:493333e9491fba57ff1ea5b8058546fc5a3e124d325f8720c018568f852e385b -o /dev/null
+ read p
+ prefix=https://[MASKED]/artifactory/api/docker/registry-gitlab-com/v2/gitlab-org/cluster-integration/auto-build-image/blobs/sha256:4db90bcd796b9a31f9366d08b929131bdc58ba070f57d3195df477af51d0c898
+ read p
+ prefix=https://[MASKED]/artifactory/api/docker/registry-gitlab-com/v2/gitlab-org/cluster-integration/auto-build-image/blobs/sha256:540db60ca9383eac9e418f78490994d0af424aab7bf6d0e47ac8ed4e2e9bcbba
+ curl -sS -u[MASKED] -w 'HTTP/1.1 %{http_code} | %{time_total} seconds | %{size_download} bytes\n' https://[MASKED]/artifactory/api/docker/registry-gitlab-com/v2/gitlab-org/cluster-integration/auto-build-image/blobs/sha256:4db90bcd796b9a31f9366d08b929131bdc58ba070f57d3195df477af51d0c898 -o /dev/null
+ read p
+ prefix=https://[MASKED]/artifactory/api/docker/registry-gitlab-com/v2/gitlab-org/cluster-integration/auto-build-image/blobs/sha256:5a38b3726f4b24fa93b80450be63ad67fd3239c2f3b83695118d7b1a88447d84
+ read p
+ curl -sS -u[MASKED] -w 'HTTP/1.1 %{http_code} | %{time_total} seconds | %{size_download} bytes\n' https://[MASKED]/artifactory/api/docker/registry-gitlab-com/v2/gitlab-org/cluster-integration/auto-build-image/blobs/sha256:540db60ca9383eac9e418f78490994d0af424aab7bf6d0e47ac8ed4e2e9bcbba -o /dev/null
+ prefix=https://[MASKED]/artifactory/api/docker/registry-gitlab-com/v2/gitlab-org/cluster-integration/auto-build-image/blobs/sha256:7732b60e038bc1bd67df0ba16b60246242be428b5342f88f38a1a2a28cea09b6
+ curl -sS -u[MASKED] -w 'HTTP/1.1 %{http_code} | %{time_total} seconds | %{size_download} bytes\n' https://[MASKED]/artifactory/api/docker/registry-gitlab-com/v2/gitlab-org/cluster-integration/auto-build-image/blobs/sha256:5a38b3726f4b24fa93b80450be63ad67fd3239c2f3b83695118d7b1a88447d84 -o /dev/null
+ read p
+ prefix=https://[MASKED]/artifactory/api/docker/registry-gitlab-com/v2/gitlab-org/cluster-integration/auto-build-image/blobs/sha256:962e97023e1ad99b2f9ef422b8965d95386a547a3c39ca8d42504054e846b2cb
+ curl -sS -u[MASKED] -w 'HTTP/1.1 %{http_code} | %{time_total} seconds | %{size_download} bytes\n' https://[MASKED]/artifactory/api/docker/registry-gitlab-com/v2/gitlab-org/cluster-integration/auto-build-image/blobs/sha256:7732b60e038bc1bd67df0ba16b60246242be428b5342f88f38a1a2a28cea09b6 -o /dev/null
+ read p
+ prefix=https://[MASKED]/artifactory/api/docker/registry-gitlab-com/v2/gitlab-org/cluster-integration/auto-build-image/blobs/sha256:e5fa5deb334027202841b051d10e7c7137fa3b63e97734309cedf6b48804df5f
+ curl -sS -u[MASKED] -w 'HTTP/1.1 %{http_code} | %{time_total} seconds | %{size_download} bytes\n' https://[MASKED]/artifactory/api/docker/registry-gitlab-com/v2/gitlab-org/cluster-integration/auto-build-image/blobs/sha256:962e97023e1ad99b2f9ef422b8965d95386a547a3c39ca8d42504054e846b2cb -o /dev/null
+ read p
+ prefix=https://[MASKED]/artifactory/api/docker/registry-gitlab-com/v2/gitlab-org/cluster-integration/auto-build-image/blobs/sha256:f75ff45c06b697172607d2d8278f8200f3d0a803e1153ab02b2751061b834175
+ curl -sS -u[MASKED] -w 'HTTP/1.1 %{http_code} | %{time_total} seconds | %{size_download} bytes\n' https://[MASKED]/artifactory/api/docker/registry-gitlab-com/v2/gitlab-org/cluster-integration/auto-build-image/blobs/sha256:e5fa5deb334027202841b051d10e7c7137fa3b63e97734309cedf6b48804df5f -o /dev/null
+ read p
+ wait
+ curl -sS -u[MASKED] -w 'HTTP/1.1 %{http_code} | %{time_total} seconds | %{size_download} bytes\n' https://[MASKED]/artifactory/api/docker/registry-gitlab-com/v2/gitlab-org/cluster-integration/auto-build-image/blobs/sha256:f75ff45c06b697172607d2d8278f8200f3d0a803e1153ab02b2751061b834175 -o /dev/null
HTTP/1.1 404 | 0.073892 seconds | 87 bytes
HTTP/1.1 404 | 0.074858 seconds | 87 bytes
HTTP/1.1 404 | 0.091974 seconds | 87 bytes
HTTP/1.1 404 | 0.082049 seconds | 87 bytes
HTTP/1.1 404 | 0.073985 seconds | 87 bytes
HTTP/1.1 404 | 0.081994 seconds | 87 bytes
HTTP/1.1 404 | 0.081283 seconds | 87 bytes
HTTP/1.1 404 | 0.084249 seconds | 87 bytes
HTTP/1.1 404 | 0.078050 seconds | 87 bytes
HTTP/1.1 404 | 0.083284 seconds | 87 bytes
HTTP/1.1 404 | 0.087073 seconds | 87 bytes
HTTP/1.1 404 | 0.087608 seconds | 87 bytes
+ rm marker_layers.txt marker_paths.txt download_markers.txt
+ [[ yes =~ [nN](o)* ]]

When I try to call one of the URLs from the terminal, I receive the following response:

❯ curl -u[MASKED] https://[MASKED]/artifactory/api/docker/registry-gitlab-com/v2/gitlab-org/cluster-integration/auto-build-image/blobs/sha256:f75ff45c06b697172607d2d8278f8200f3d0a803e1153ab02b2751061b834175 
{
  "errors" : [ {
    "status" : 404,
    "message" : "authenticationrequired"
  } ]
}%