Create ECS tasks definitions from the CICD (github-actions)

[tloubrieu-jpl]

💡 Description

Define the proper revision scheme related to the actual version of the components. That could be using the version of the components or tags (latest, stable...)

Applies to the following repositories:

For registry:

• [ ] registry-api
• [ ] registry-sweepers

For nucleus:

• [ ] validate
• [ ] deep-archive
• [ ] registry-loader

Validate that the tasks can be used on the AWS deployments (e.g. en-delta for registry).

[sjoshi-jpl]

@tloubrieu-jpl what is the scope of this task? Are we creating new tasks definitions for nucleus (registry-api / registry-sweeper task definitions already exists) or am I just setting up their deployment in NGAP using GitHub Actions?

Also, can you please provide further information regarding this statement in the description above, "Define the proper revision scheme related to the actual version of the components".

[tloubrieu-jpl]

We need to consider the traceability of the deployment by tracking:

• commit id in github when a merge is done on main
• packages (jar, pypi), snapshot, dev --> commit number (@nutjob4life )
• docker image version, latest --> commit number (@nutjob4life )
• ECS task defintinion, revision number contains the description of the docker image

This is not in the scope of this ticket. @tloubrieu-jpl will create a new ticket for that.

[sjoshi-jpl]

@tloubrieu-jpl for registry-api and registry-sweeper workflows, do we want the workflow to deploy the services to all OpenSearch domains or just 1 domain since that's the architecture we're moving towards. I'll setup the workflow accordingly, because if we do all domains then I'll have to either ask user for input values for variables specific to each domain or setup a plan / apply for each domain (so repeat the tasks 12 times which is not efficient).

[tloubrieu-jpl]

@sjoshi-jpl I think we can deploy on a single domain, which will be synonym for venue (stage dev, stage I&T or prod) in the upcoming configuration. On NGAP we only have the stage dev venue.

[sjoshi-jpl]

@tloubrieu-jpl sounds good, I'll create a workflow for a single domain.

I had a discussion with @ramesh-maddegoda regarding this task as well, and based on his feedback for Nucleus components, I'll be creating a worfklow in the registry-loader repo since validate and deep-archive currently do not have terraform ecs tasks configured.

Once @ramesh-maddegoda is able to configure tasks for those services using terraform, I'll add the workflow in those repos as well.

[sjoshi-jpl]

@tloubrieu-jpl Forgot to mention this but creating the terraform-workflow will trigger a terraform apply on merge to main. So currently the workflow I am setting up is for NGAP DEV since we cannot deploy to AWS cloud. Let me know if thats a concern.

[jordanpadams]

@sjoshi-jpl working on Nucleus deployments

[sjoshi-jpl]

@tloubrieu-jpl can you please give me admin rights to the above mentioned repos? I will need to configure repo specific variables for each

[ramesh-maddegoda]

Created Terraform scripts and documentation to deploy ECS Task Definition for PDS Validate Tool https://github.com/NASA-PDS/validate/pull/672

[nutjob4life]

@ramesh: created ECS task for validate, and it was merged (a terraform script). Next up: Sagar will be up.