Add support for RSI configs in import automation

datacommonsorg / data

Apache License 2.0

60 stars 107 forks source link

Closed ajaits closed 6 months ago

ajaits commented 6 months ago

Add support for RSI in Import automation:

manifests with scripts under 'scripts/simple/../simple_config.json' are treated as RSI imports
to run such imports as a cloud run job
- copy config to GCS
- create a versioned output directory on GCS
- launch cloud run with CONFIG and OUTPUT_DIR env variables