search

notAI-tech / NudeNet

Lightweight nudity detection
https://nudenet.notai.tech/
GNU Affero General Public License v3.0
1.73k stars 337 forks source link

Are classify and detect reentrant functions? #78

Open patrickjane opened 3 years ago

patrickjane commented 3 years ago

First of all, thanks a ton for this awesome piece of software, much appreciated!

I would just like to ask a quick question. Are the classify and detect functions reentrant? E.g. is it safe to use those from a webserver's request handlers, and thus potentially having multiple classifications running at the same time?

Note: Currently not using the docker image, using a plain python install with my own webserver wrapper.

bedapudi6788 commented 3 years ago

You can have multiple classification running in different threads/ processes with no problem. BUT, the ideal way to efficiently run multiple images/predictions at the same time is batch inference. https://github.com/notAI-tech/fastDeploy (using which the docker images are built) is optimized for this.

patrickjane commented 3 years ago

Okay cool.

Batch is not an option, since I want to classify user uploads, and thus, it must be classified on-request.

bedapudi6788 commented 3 years ago

fastDeploy does support micro batching on user-requiest via /sync. If 2 requests come in at the same time, fastDeploy batch predicts them and returns the response in the same http call.

For example, take a look at https://fastdeploy.notai.tech/docs/benchmarks.html#deepsegment_en

8192 requests when run one by one takes 79.06 seconds. If you have 128 users running total of 8192 requests one by one this would take only 7.05 seconds (10x performance increase in sync because of batching.)

bedapudi6788 commented 3 years ago

Take a look at https://fastdeploy.notai.tech/api#request-type-file

patrickjane commented 3 years ago

Sounds great, will investigate.