aws / aws-sdk

Landing page for the AWS SDKs on GitHub
https://aws.amazon.com/tools/
Other
72 stars 14 forks source link

AWS CodeArtifact package lifecycle: add publishedTime attribute to PackageVersionSummary object #596

Closed davimmt closed 1 year ago

davimmt commented 1 year ago

Describe the feature

When we call the ListPackageVersions operation, it returns a list of PackageVersionSummary objects. This object is pretty much like the PackageVersionDescription, however with less attributes. I'd like to include the PackageVersionDescription .packageVersion.publishedTime to the PackageVersionSummary .versions.

Use Case

When writing scripts to clean/purge packages (manging a custom lifecycle for artifacts/packages), let's say we want to have only the 3 latests packages versions. In order to delete older versions, we have to loop through packages to get packages versions and then loop through packages versions (ListPackageVersions) to get the package versions details (PackageVersionDescription) and find the publishedTime in order to sort all packages. Problem is, this requires a API describe call to every and each version, and it takes a long time depending on how much versions you have for every single artifact package.

Proposed Solution

I propose to include the key .packageVersion.publishedTime from the PackageVersionDescription to the object PackageVersionSummary (.versions.publishedTime).

Now, the aws codeartifact list-package-versions outputs as follows:

{
  "versions": [
    {
      "version": "1.0.0-STAGING.11",
      "revision": "XXXXXXXXXXXXXXXXXXXX",
      "status": "Published",
      "origin": {
        "domainEntryPoint": {
          "repositoryName": "npm-store"
        },
        "originType": "INTERNAL"
      }
    },
    {
      "version": "1.1.0",
      "revision": "XXXXXXXXXXXXXXXXXXXX",
      "status": "Published",
      "origin": {
        "domainEntryPoint": {
          "repositoryName": "npm-store"
        },
        "originType": "INTERNAL"
      }
    }
  ],
  "defaultDisplayVersion": "1.1.0",
  "format": "npm",
  "package": "XXXXXXXXXXXXXXXXXXXX",
  "namespace": "XXXXXXXXXXXXXXXXXXXX"
}

And I can sort the packages by published time using:

for $p in $(aws codeartifact list-packages $OPTS | jq -r '.packages[].package'); do
  for $pv in $(aws codeartifact list-package-versions $PACKAGE_OPTS | jq -r '.versions[].version'); do
    aws codeartifact describe-package-version $PACKAGE_VERSION_OPTS | jq '.packageVersion.publishedTime'
  done
done

Propose:

{
  "versions": [
    {
      "version": "1.0.0-STAGING.11",
      "revision": "XXXXXXXXXXXXXXXXXXXX",
      "status": "Published",
      "publishedTime": "2023-08-15T12:52:39.864000-03:00",
      "origin": {
        "domainEntryPoint": {
          "repositoryName": "npm-store"
        },
        "originType": "INTERNAL"
      }
    },
    {
      "version": "1.1.0",
      "revision": "XXXXXXXXXXXXXXXXXXXX",
      "status": "Published",
      "publishedTime": "2023-08-15T12:52:39.864000-03:00",
      "origin": {
        "domainEntryPoint": {
          "repositoryName": "npm-store"
        },
        "originType": "INTERNAL"
      }
    }
  ],
  "defaultDisplayVersion": "1.1.0",
  "format": "npm",
  "package": "XXXXXXXXXXXXXXXXXXXX",
  "namespace": "XXXXXXXXXXXXXXXXXXXX"
}

With this output, we could sort just by (sparing a lot of API calls):

for $p in $(aws codeartifact list-packages $OPTS | jq -r '.packages[].package'); do
  aws codeartifact list-package-versions $PACKAGE_OPTS | jq '.versions[].publishedTime'
done

Other Information

No response

Acknowledgements

CLI version used

aws-cli/2.12.6 Python/3.11.4 Linux/5.15.90.1-microsoft-standard-WSL2 exe/x86_64.ubuntu.20 prompt/off

Environment details (OS name and version, etc.)

Linux DM-0028 5.15.90.1-microsoft-standard-WSL2 aws/aws-cli#1 SMP Fri Jan 27 02:56:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

RyanFitzSimmonsAK commented 1 year ago

Hi @davimmt, thanks for reaching out. Because this is a request for a change to a service, it would need to be directed towards the service team. I've reached out to the Code Artifact team to ask about this feature request, and I'll leave any updates in this issue. I'm also going to move it to our cross-SDK repository (https://github.com/aws/aws-sdk/). Thanks!

RyanFitzSimmonsAK commented 1 year ago

Ticket # for internal use : P98568505

davimmt commented 1 year ago

Just some information about the use case: a few of our teams are migrating to AWS Code Artifact and the 'purger' pipeline outputs:

We have [58] packages in total.
.
.
.
Excluded a total of [194] packages versions from a total of [585] package versions. 
This operantion took [16m 48s]. 
DRYRUN is set to [true], so nothing was deleted.
RyanFitzSimmonsAK commented 1 year ago

Hi @davimmt, thanks for your patience. The Code Artifact team is open to considering this feature request, but don't have a specific timeline for it currently. They did suggest using --sort-by PUBLISHED_TIME as a potential workaround (see docs). Hope that helps!

github-actions[bot] commented 1 year ago

This issue is now closed.

Comments on closed issues are hard for our team to see. If you need more assistance, please either tag a team member or open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.

davimmt commented 1 year ago

That is very helpful, thank you so much. I have now removed the describe-package-version for loop, and so far it works as expected with a good runtime.

image