search

exorde-labs / Exorde-Client-Microservice-Mint

Micro-Service based Exorde-Client
https://exorde.io/
1 stars 2 forks source link

Horizontal Scaling on Bpipe #1

Open 6r17 opened 7 months ago

6r17 commented 7 months ago

In order to handle a batch as fast as possible the pipe requires to be filled appropriately. The balancing algorithm for it is complex and require orchestration, and time spent on it, additionally to any ops related configuration that would let users attribute GPU resources dynamically.

This might be added to a TODO list but since the other option is to manage the stack with a single GPU, or there might be others I don't see yet, I'm not spending time on it before it get's attention or is something the community asks for.

epsilonion-liam commented 3 months ago

Scaling bpipe would be awesome, not everyone has a GPU in there server.. and to scale for CPU on bpipe would be amazing.

users are cutting back there workers with the price of electricity been high at the moment (as mentioned on discord), my server is pushing 600w consumption at the moment and is not sustainable (UK based) as electricity is expensive here, been able to push bpipe onto a device dedicated to it and scalable on bpipe would be an advantage.