search

speechmatics / artifactory-user-plugins

Sample Artifactory User Plugins
http://wiki.jfrog.org/confluence/display/RTF/User+Plugins
Apache License 2.0
0 stars 0 forks source link

cleanDockerImages: byDownloadDate cleanup mode #1

Closed weakcamel closed 2 years ago

weakcamel commented 2 years ago

A draft only pull request to be able to review the changes more easily.

Implements byDownloadDate mode of operation - see https://github.com/jfrog/artifactory-user-plugins/issues/333

weakcamel commented 2 years ago

Tests for maxDays option

Tests data (input)

docker-test/test/v1/manifest.json creat=2022-01-17T10:08:28.118Z modified=2022-01-17T10:10:17.194Z maxDays=1
docker-test/test/v2/manifest.json creat=2022-01-17T10:12:18.444Z modified=2022-01-17T10:12:18.439Z maxDays=1
docker-test/test/v3/manifest.json creat=2022-01-17T10:14:19.751Z modified=2022-01-17T10:14:19.747Z maxDays=1
docker-test/test/v4/manifest.json creat=2022-01-17T10:16:21.080Z modified=2022-01-17T10:16:21.075Z maxDays=1
docker-test/test/v5/manifest.json creat=2022-01-17T10:18:22.315Z modified=2022-01-17T10:18:22.311Z maxDays=1
docker-test/test/v6/manifest.json creat=2022-01-17T10:20:23.568Z modified=2022-01-17T10:20:23.564Z maxDays=1
docker-test/test/v7/manifest.json creat=2022-01-17T10:22:24.817Z modified=2022-01-17T10:22:24.813Z maxDays=1
docker-test/test/v8/manifest.json creat=2022-01-17T10:24:26.491Z modified=2022-01-17T10:24:26.486Z maxDays=1
docker-test/test/v9/manifest.json creat=2022-01-17T10:28:01.737Z modified=2022-01-17T10:28:01.731Z maxDays=1

Out of these images, none have been downloaded except for:

Test output:

+ /usr/bin/curl -sSfL -X POST -u admin:<REDACTED> 'http://localhost:8081/artifactory/api/plugins/execute/cleanDockerImages?params=dryRun=true'
+ jq .
{
  "status": "okay",
  "dryRun": true,
  "deleted": [
    "docker-test:test/v1",
    "docker-test:test/v2",
    "docker-test:test/v3",
    "docker-test:test/v4",
    "docker-test:test/v5",
    "docker-test:test/v6",
    "docker-test:test/v7",
    "docker-test:test/v8",
    "docker-test:test/v9"
  ]
}
Trigger the plugin - byDownloadDate mode
+ /usr/bin/curl -sSfL -X POST -u admin:<REDACTED> 'http://localhost:8081/artifactory/api/plugins/execute/cleanDockerImages?params=dryRun=true;byDownloadDate=true'
+ jq .
{
  "status": "okay",
  "dryRun": true,
  "deleted": [
    "docker-test:test/v1",
    "docker-test:test/v2",
    "docker-test:test/v3",
    "docker-test:test/v4",
    "docker-test:test/v6",
    "docker-test:test/v7",
    "docker-test:test/v9"
  ]
}
weakcamel commented 2 years ago

Tests for maxCount option

Test script

#!/bin/bash
set -eEuo pipefail

for i in $(seq 1 9 | tac); do
  echo "- - - - - - - - - - - - - - - - - - - - - - - - - - -"
  echo "Changing maxCount to ${i}"
  ./jfrog rt set-props docker-test/test/**/manifest.json "docker.label.com.jfrog.artifactory.retention.maxCount=${i}" >/dev/null 2>&1
  #echo "Trigger the plugin - default mode"
  #./jfrog rt curl -sSfL -X POST "api/plugins/execute/cleanDockerImages?params=dryRun=true" | jq -r ".deleted | .[]"
  echo "Trigger the plugin - byDownloadDate mode"
  ./jfrog rt curl -sSfL -X POST "api/plugins/execute/cleanDockerImages?params=dryRun=true;byDownloadDate=true" | jq -r ".deleted | .[]"
done

Input data

Image versions v1 - v9 were created in order (v1 oldest to v9 latest)

docker-test/test/v1/manifest.json creat=2022-01-17T10:08:28.118Z modified=2022-01-17T10:10:17.194Z maxDays=10
docker-test/test/v2/manifest.json creat=2022-01-17T10:12:18.444Z modified=2022-01-17T10:12:18.439Z maxDays=10
docker-test/test/v3/manifest.json creat=2022-01-17T10:14:19.751Z modified=2022-01-17T10:14:19.747Z maxDays=10
docker-test/test/v4/manifest.json creat=2022-01-17T10:16:21.080Z modified=2022-01-17T10:16:21.075Z maxDays=10
docker-test/test/v5/manifest.json creat=2022-01-17T10:18:22.315Z modified=2022-01-17T10:18:22.311Z maxDays=10
docker-test/test/v6/manifest.json creat=2022-01-17T10:20:23.568Z modified=2022-01-17T10:20:23.564Z maxDays=10
docker-test/test/v7/manifest.json creat=2022-01-17T10:22:24.817Z modified=2022-01-17T10:22:24.813Z maxDays=10
docker-test/test/v8/manifest.json creat=2022-01-17T10:24:26.491Z modified=2022-01-17T10:24:26.486Z maxDays=10
docker-test/test/v9/manifest.json creat=2022-01-17T10:28:01.737Z modified=2022-01-17T10:28:01.731Z maxDays=10

Out of these images, only these 2 have been downloaded:

Test output for byDownloadDate

root@artifactory:~# ./trigger_clean.sh
- - - - - - - - - - - - - - - - - - - - - - - - - - -
Changing maxCount to 9
Trigger the plugin - byDownloadDate mode
- - - - - - - - - - - - - - - - - - - - - - - - - - -
Changing maxCount to 8
Trigger the plugin - byDownloadDate mode
docker-test:test/v1
- - - - - - - - - - - - - - - - - - - - - - - - - - -
Changing maxCount to 7
Trigger the plugin - byDownloadDate mode
docker-test:test/v1
docker-test:test/v2
- - - - - - - - - - - - - - - - - - - - - - - - - - -
Changing maxCount to 6
Trigger the plugin - byDownloadDate mode
docker-test:test/v1
docker-test:test/v2
docker-test:test/v3
- - - - - - - - - - - - - - - - - - - - - - - - - - -
Changing maxCount to 5
Trigger the plugin - byDownloadDate mode
docker-test:test/v1
docker-test:test/v2
docker-test:test/v3
docker-test:test/v4
- - - - - - - - - - - - - - - - - - - - - - - - - - -
Changing maxCount to 4
Trigger the plugin - byDownloadDate mode
docker-test:test/v1
docker-test:test/v2
docker-test:test/v3
docker-test:test/v4
docker-test:test/v6
- - - - - - - - - - - - - - - - - - - - - - - - - - -
Changing maxCount to 3
Trigger the plugin - byDownloadDate mode
docker-test:test/v1
docker-test:test/v2
docker-test:test/v3
docker-test:test/v4
docker-test:test/v6
docker-test:test/v7
- - - - - - - - - - - - - - - - - - - - - - - - - - -
Changing maxCount to 2
Trigger the plugin - byDownloadDate mode
docker-test:test/v1
docker-test:test/v2
docker-test:test/v3
docker-test:test/v4
docker-test:test/v6
docker-test:test/v7
docker-test:test/v9
- - - - - - - - - - - - - - - - - - - - - - - - - - -
Changing maxCount to 1
Trigger the plugin - byDownloadDate mode
docker-test:test/v1
docker-test:test/v2
docker-test:test/v3
docker-test:test/v4
docker-test:test/v6
docker-test:test/v7
docker-test:test/v9
docker-test:test/v5
root@artifactory:~#

Note the 2 last cases: image v5 was identified for removal as last and the last downloaded image v8 was not marked for removal at all.

For reference - test output for the default mode (by creation date - existing plugin behaviour).

oot@artifactory:~# ./trigger_clean.sh
- - - - - - - - - - - - - - - - - - - - - - - - - - -
Changing maxCount to 9
Trigger the plugin - default mode
- - - - - - - - - - - - - - - - - - - - - - - - - - -
Changing maxCount to 8
Trigger the plugin - default mode
docker-test:test/v1
- - - - - - - - - - - - - - - - - - - - - - - - - - -
Changing maxCount to 7
Trigger the plugin - default mode
docker-test:test/v1
docker-test:test/v2
- - - - - - - - - - - - - - - - - - - - - - - - - - -
Changing maxCount to 6
Trigger the plugin - default mode
docker-test:test/v1
docker-test:test/v2
docker-test:test/v3
- - - - - - - - - - - - - - - - - - - - - - - - - - -
Changing maxCount to 5
Trigger the plugin - default mode
docker-test:test/v1
docker-test:test/v2
docker-test:test/v3
docker-test:test/v4
- - - - - - - - - - - - - - - - - - - - - - - - - - -
Changing maxCount to 4
Trigger the plugin - default mode
docker-test:test/v1
docker-test:test/v2
docker-test:test/v3
docker-test:test/v4
docker-test:test/v5
- - - - - - - - - - - - - - - - - - - - - - - - - - -
Changing maxCount to 3
Trigger the plugin - default mode
docker-test:test/v1
docker-test:test/v2
docker-test:test/v3
docker-test:test/v4
docker-test:test/v5
docker-test:test/v6
- - - - - - - - - - - - - - - - - - - - - - - - - - -
Changing maxCount to 2
Trigger the plugin - default mode
docker-test:test/v1
docker-test:test/v2
docker-test:test/v3
docker-test:test/v4
docker-test:test/v5
docker-test:test/v6
docker-test:test/v7
- - - - - - - - - - - - - - - - - - - - - - - - - - -
Changing maxCount to 1
Trigger the plugin - default mode
docker-test:test/v1
docker-test:test/v2
docker-test:test/v3
docker-test:test/v4
docker-test:test/v5
docker-test:test/v6
docker-test:test/v7
docker-test:test/v8
root@artifactory:~#