opendatacube / datacube-wps

Web Processing Service running on opendatacube

Other

7 stars 3 forks source link

improve docker-compose and fix test ci #138

Closed pindge closed 2 years ago

pindge commented 2 years ago

remove indexer service from docker-compose.yaml
add healthcheck to postgres service in docker-compose.yaml
remove legacy env setting from test ci
improve wps depends_on for postgres readiness
remove trailing whitespace from db-setup.sh
pin test ci to only run again master branch

codecov[bot] commented 2 years ago

Codecov Report

Merging #138 (e634158) into master (bc38b41) will not change coverage. The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #138   +/-   ##
=======================================
  Coverage   66.25%   66.25%           
=======================================
  Files           8        8           
  Lines         643      643           
=======================================
  Hits          426      426           
  Misses        217      217

benjimin commented 2 years ago

I agree that using healthcheck/dependency correctly is a solution to our main problem, of the runner executing setup-db.sh before the postgres service is ready. However, this solution assumes that docker-compose up -d will not return until all services are started (specifically including the WPS container, which seems a slightly convoluted way to force the runner to wait for the postgres service) and this seems to be an undocumented behaviour. Maybe we should refactor the script to run from inside the WPS container, to guarantee the behaviour?
I think test CI should still run against all branches, not just master. (I think its purpose is to help catching bugs before they propagate into the main branch.)
The setup-db.sh script will need updating to support removal of the indexer container. (It will periodically be necessary to remove the SQL dump file and let the script regenerate it, due to either new WPS tests or DEA collection reorganisation.)

benjimin commented 2 years ago

Update:

Updated setup-db.sh for indexer deprecation c449e6f3
Applied catalog env var deprecation c100646
Ensure setup-db.sh invoked after db ready to accept connections 6c3846d2
Continuing to let all branches be tested

CI now passing.

Thanks @pindge for all the assistance!