globus / globus-compute

Globus Compute: High Performance Function Serving for Science
https://www.globus.org/compute
Apache License 2.0
138 stars 44 forks source link

Use endpoint CLI to launch worker pool #1600

Open rjmello opened 6 days ago

rjmello commented 6 days ago

Description

This PR adds the globus-compute-endpoint python-exec command to run Python modules as scripts via the Globus Compute endpoint CLI.

E.g.,

globus-compute-endpoint python-exec parsl.executors.high_throughput.process_worker_pool --debug

We then use this to launch worker pools without requiring worker nodes to resolve the process_worker_pool.py command.

[sc-34376]

Type of change