search

ShannonAI / service-streamer

Boosting your Web Services of Deep Learning Applications.
Apache License 2.0
1.22k stars 187 forks source link

Problem using service_streamer on gunicorn setup #63

Open vijay-tp opened 4 years ago

vijay-tp commented 4 years ago

Hey, Thank you for an wonderful code. I'm trying to use service_streamer on my application. My application has deployed into gunicorn server. When using service_streamer into my application, it couldn't able to batch the requests. But it works when using streamer into outside gunicorn setup. My streamer code is, streamer = Streamer(self.corrector.predict, batch_size=64) The log is shown in the below., INFO:service_streamer.service_streamer:[gpu worker 30430] run_once batch_size: 1 start_at: 1586233589.1705348 spend: 0.18162846565246582 Actually the batch size is 64, but it running with batch size 1. If you come across this problem, it will be really helpful your suggestions! Thanks in Advance :)

Meteorix commented 4 years ago

take a look at this example: https://github.com/ShannonAI/service-streamer/blob/master/example/redis_streamer_gunicorn.py