search

awslabs / autonomous-driving-data-framework

ADDF is a collection of modules, deployed using the SeedFarmer orchestration tool. ADDF modules enable users to quickly bootstrap environments for the process and analysis of autonomous driving data.
Apache License 2.0
113 stars 44 forks source link

[FEATURE] CI/CD Pipeline for testing deployment and destruction of manifests #450

Closed a13zen closed 6 months ago

a13zen commented 7 months ago

Is your feature request related to a problem? Please describe. We've been encountering issues where changes such as deprecated lambda runtimes, dependencies that have been archived due to CVEs etc. have caused some modules to fail. Also changes upstream from IDF/AIOps Modules sometimes have unexpected side effects on older deployments.

Describe the solution you'd like

Integrate GitHub Actions using the new support for self hosted runners on CodeBuild to automatically deploy + destroy all deployments under the manifests folder. This should allow us to get clear, early indications of issues with modules.

https://docs.aws.amazon.com/codebuild/latest/userguide/action-runner.html

This also would allow us to test the deployspec.yaml for creation and destruction and catch any potential issues.

malachi-constant commented 7 months ago

We have this mostly done for internal testing. I can put together a module that would deploy a similar component that would run integration testing for any list of manifests.

a13zen commented 7 months ago

This is great. Obviously it would be great if this was a module that would be usable by anyone for their SF based deployments. However, the main ask here is for us to detect and be notified early whenever manifests (and downstream their modules) are having any issues deploying/destroying.

malachi-constant commented 6 months ago

This is great. Obviously it would be great if this was a module that would be usable by anyone for their SF based deployments. However, the main ask here is for us to detect and be notified early whenever manifests (and downstream their modules) are having any issues deploying/destroying.

Yep, so I have that idea implemented here: https://github.com/awslabs/idf-modules/pull/174

User would be able to deploy inline within their manifest or deploy this module separately via CDK and/or seedfarmer. Then any changes checked-in to the designated repo/branch would trigger the E2E tests of any specified manifests.

malachi-constant commented 6 months ago

This is now available via IDF: https://github.com/awslabs/idf-modules/tree/main/modules/testing/integration-tests

I think this is a viable starting point for the functionality called out above. We can continue to enhance as needed.