GSA / fedramp-automation

FedRAMP Automation
https://www.fedramp.gov/using-the-fedramp-oscal-resources-and-templates/
Other
284 stars 87 forks source link

Content processing CI/CD is flaky #643

Open aj-stein-gsa opened 2 months ago

aj-stein-gsa commented 2 months ago

This relates to ...

What happened?

It seems the content generation and testing CI in GHA is flaky looking at #641 failing on an unrelated change and clicking back through the master branch from time to time.

Recent GHA job output leads me to one hypothesis, that is some parts of how to run Docker Compose has changed and the old way is no longer supported.

In the last few years, Docker Compose had a single wrapper command executable, docker-compose and then moved to docker compose (observe the space). It would seem that GitHub discussion chatter confirms that was obsoleted.

This would mean local tests work, but not GHA. We need to investigate 1) if this is the only root case; 2) confirm/deny other root causes; 3) fix some/none/all and improve stability of all tests on PR and post-merge.

Relevant log output

##[debug]Evaluating condition for step: 'Build content generator'
##[debug]Evaluating: success()
##[debug]Evaluating success:
##[debug]=> true
##[debug]Result: true
##[debug]Starting: Build content generator
##[debug]Loading inputs
##[debug]Loading env
Run make init-content
##[debug]/usr/bin/bash -e /home/runner/work/_temp/4b01530c-f811-45dd-82e9-e2bd38a89b21.sh
Building Docker image for OSCAL content generation...
docker-compose -f vendor/oscal/build/docker-compose.yml -f src/content/docker-compose.yml build
make: docker-compose: Command not found
make: *** [src/content/module.mk:16: init-content] Error 127
Error: Process completed with exit code 2.
##[debug]Finishing: Build content generator

How do we replicate this issue?

  1. Create a PR with any change.
  2. Observe somewhat consistent failure of the content-artifacts.yml workflow.

Where, exactly?

GitHub Actions, so .github/workflows.

Other relevant details

No response

aj-stein-gsa commented 1 month ago

I actually "completed the work," but it is not merged up to master yet so I will leave this open and mark "in progress" until it is merged there and complete.