NASA-IMPACT / veda-pforge-job-runner

Apache Beam + EMR Serverless Job Runner for Pangeo Forge Recipes
2 stars 2 forks source link

Passing: LEAP PFG #14

Closed ranchodeluxe closed 4 months ago

ranchodeluxe commented 8 months ago
pangeo-forge-runner bake \
    --repo=https://github.com/ranchodeluxe/leap-pgf-example \
    --ref="main" \
    -f config.py \
    --Bake.job_name=agcd
pangeo-forge-runner bake \
    --repo=https://github.com/ranchodeluxe/leap-pgf-example \
    --ref="main" \
    -f config.py \
    --Bake.job_name=agcd
curl -X POST \                                                                   
 -H "Accept: application/vnd.github+json" \
 -H "X-GitHub-Api-Version: 2022-11-28" \
 -H "Authorization: token blahblah" \
 https://api.github.com/repos/NASA-IMPACT/veda-pforge-job-runner/actions/workflows/job-runner.yaml/dispatches \
 -d '{"ref":"main", "inputs":{"repo":"https://github.com/ranchodeluxe/leap-pgf-example","ref":"main","prune":"1"}}'
curl -X POST \                                                                   
 -H "Accept: application/vnd.github+json" \
 -H "X-GitHub-Api-Version: 2022-11-28" \
 -H "Authorization: token blahblah" \
 https://api.github.com/repos/NASA-IMPACT/veda-pforge-job-runner/actions/workflows/job-runner.yaml/dispatches \
 -d '{"ref":"main", "inputs":{"repo":"https://github.com/ranchodeluxe/leap-pgf-example","ref":"main","prune":"0"}}'
ranchodeluxe commented 8 months ago

last prune run failed: https://github.com/NASA-IMPACT/veda-pforge-job-runner/actions/runs/7441506768

ranchodeluxe commented 8 months ago

last prune run failed: https://github.com/NASA-IMPACT/veda-pforge-job-runner/actions/runs/7441506768

reviewing the logs again with some monocles reveals an reason=OOMKilled so we'll need a bigger nodes it seems

ranchodeluxe commented 8 months ago

to get even the prune option working I had to allocate 10GB of memory to the task manager 😬

norlandrhagen commented 8 months ago

Oof