NASA-IMPACT / veda-pforge-job-runner

Apache Beam + EMR Serverless Job Runner for Pangeo Forge Recipes
2 stars 2 forks source link

Before Recipes Are Parsed Runner Should Create Temp Venv with Recipe Requirements #24

Closed ranchodeluxe closed 6 months ago

ranchodeluxe commented 7 months ago

Problem

Some recipes need to run logic (which has certain deps) on the host runner and not the worker. Regardless, the runner shouldn't have any surprise deps to kick off work that aren't listed in the pyproject.toml. Really this is just to track development of https://github.com/pangeo-forge/pangeo-forge-runner/issues/154

Solution

Work on the things mentioned herez: https://github.com/pangeo-forge/pangeo-forge-runner/issues/154

ranchodeluxe commented 6 months ago

doneso!