als-computing / splash_flows_globus

Configuration and code for Prefect workflows to move data and run computing tasks
Other
1 stars 3 forks source link

Separate 832 move.py new_file and pruning work queues, set priorities #38

Open davramov opened 1 month ago

davramov commented 1 month ago

Currently one work pool on flow-prd for bl832, default-agent-pool with the queue named bl832, is responsible for handling these deployed functions: new_file_832, prune_data832 and prune_spot832. We should separate this work into a second queue under the work pool called bl832-pruning that takes care pruning tasks. Once they are in separate pools, we can set new_file_832 to be high priority, and prune flows to be low priority.