search

ShannonAI / service-streamer

Boosting your Web Services of Deep Learning Applications.
Apache License 2.0
1.22k stars 187 forks source link

为什么我的batch_size无论设多大，处理时一次最多就接收10个 #95

Closed Kuzhuahu closed 1 year ago

Kuzhuahu commented 1 year ago

如题

cgr71ii commented 1 year ago

Hi! @Kuzhuahu were you able to solve this problem? I doesn't matter the batch size I configure I can't increase the allocated GPU memory.

cgr71ii commented 1 year ago

Solved! I just needed to increase the value of max_latency in ThreadedStreamer.

I also needed to use Gunicorn and set --threads to the value of my batch size in order to process all of them at once. I guess it would work with a lower value of threads too.