Kubernaut backend has a number of test layers and Forge makes it difficult to test all of them:
func tests for things like API request / response parsing and handling.
func tests for infrastructure management w/o talking to actual backends (mocks).
integration tests for infrastructure management that depends on AWS.
The first two categories are easy to run and are invoked as part of the normal forge deploy command. I exclude the third category from running for two reasons:
They are slow tests (need to setup / teardown infrastructure)
Generally not needed during normal development workflows BUT are needed when hacking on AWS code or releasing from CI (because if developers aren't running them then something SHOULD run them.)
There are is also another problem:
In order to run these tests the test code needs credentials (~/.aws/config and ~/.aws/credentials), however, I do not want to provide these too the Docker image being built as that will cause the credentials to be baked onto the filesystem of the image which is not desirable.
Currently my only solution is to keep the AWS tests outside of the Docker command being run during forge deploy this means the command needs to be run out of band. While that's not the end of the world functionally:
I have to rebuild the project twice... once for the out of band run and then a second time in the container for the other test runs.
What I am thinking of an improved flow around image builds as a series of independent atomic steps:
Step 1 = Base Image Build
compile (for compiled languages)
unit tests (if any)
package (convert to jar, pex, whatever)
After step <1> we're effectively in the same place we would be using the current forge build
What I want to do after this is define docker run ... commands against the image that was created in <1>. This mechanism is intended for running tests but it could actually be more generalized as a series of post-processor steps that could do anything from run tests to reduce the image size.
For example for Kubernaut a second step:
docker run -v ${HOME}/.aws:${HOME}/.aws IMAGE ./gradlew test -PrunAwsTests
Finally afterwards the image from Step 1 is pushed to the remote registry upon all subsequent post processing steps completing.
I would like this mechanism to be configurable depending on environment. For rapid development being able to just execute Step 1 but ignore all subsequent steps is fine. However, from CI, upon detecting an env var or something it would be nice if I could conditionally enable or disable various steps.
Kubernaut backend has a number of test layers and Forge makes it difficult to test all of them:
The first two categories are easy to run and are invoked as part of the normal
forge deploycommand. I exclude the third category from running for two reasons:There are is also another problem:
In order to run these tests the test code needs credentials (
~/.aws/configand~/.aws/credentials), however, I do not want to provide these too the Docker image being built as that will cause the credentials to be baked onto the filesystem of the image which is not desirable.Currently my only solution is to keep the AWS tests outside of the Docker command being run during
forge deploythis means the command needs to be run out of band. While that's not the end of the world functionally:I have to rebuild the project twice... once for the out of band run and then a second time in the container for the other test runs.
What I am thinking of an improved flow around image builds as a series of independent atomic steps:
Step 1 = Base Image Build
After step <1> we're effectively in the same place we would be using the current
forge buildWhat I want to do after this is define
docker run ...commands against the image that was created in <1>. This mechanism is intended for running tests but it could actually be more generalized as a series of post-processor steps that could do anything from run tests to reduce the image size.For example for Kubernaut a second step:
docker run -v ${HOME}/.aws:${HOME}/.aws IMAGE ./gradlew test -PrunAwsTestsFinally afterwards the image from
Step 1is pushed to the remote registry upon all subsequent post processing steps completing.I would like this mechanism to be configurable depending on environment. For rapid development being able to just execute Step 1 but ignore all subsequent steps is fine. However, from CI, upon detecting an env var or something it would be nice if I could conditionally enable or disable various steps.