Hi there, does Petals currenly support batch processing/parallel processing? For example, to increase resource usage or system throughput, we would like to see servers parallelly processing multiple prompts at the same time, aka batch processing. Is this possible?
Thanks a lot.
Hi there, does Petals currenly support batch processing/parallel processing? For example, to increase resource usage or system throughput, we would like to see servers parallelly processing multiple prompts at the same time, aka batch processing. Is this possible? Thanks a lot.