search

kube-HPC / hkube

🐟 High Performance Computing over Kubernetes - Core Repo 🎣
http://hkube.io
MIT License
307 stars 20 forks source link

Pipeline with conditional algorithms (if statements) #1173

Closed mattanf2 closed 2 years ago

mattanf2 commented 3 years ago

I have cases where I have pipelines with algorithms that may or may not need to run depending on the input of another algorithm. The way that I currently solve this is by defining those algorithms to run in a batch then either supplying an input array of 1 or an input array of 0. This is problematic for 2 reasons:

  1. Its ocward to write
  2. In a lot of cases I don't need to run the algorithm in the "if statement" but hkube loads to as a pod anyway.

A more straight forward notation of "run algo if input exists" would be better, It would also help If I could state whether the algorithm would run most times to tell hkube whether it should preload the algorithm or not.

maty21 commented 2 years ago

after thinking we decided that it's not something that we want to add to the current roadmap