search

astronomer / astro-provider-ray

This provider contains operators, decorators and triggers to send a ray job from an airflow task
https://astronomer.github.io/astro-provider-ray/
Apache License 2.0
12 stars 2 forks source link

Ability to run specific Ray jobs on specific nodes #46

Open venkatajagannath opened 2 months ago

venkatajagannath commented 2 months ago

This feature may already be possible using two method --

  1. Labelling the node group while spinning up the K8 clyster and using the resources dictionary input parameter in the SubmitRayJob operator
  2. Using placement groups in Ray

We need to research and try out both