search

datacurve-ai / image-autotagger

A web service to analyze images and tag it according to its attributes. Try it in the link below:
https://autotagger.aibooru.online/
0 stars 1 forks source link

Optimization: Batch Processing #1

Open lsha0730 opened 1 month ago

lsha0730 commented 1 month ago

We want to implement batch processing for the image classifier and thus increase throughput.

  1. Decouple the web server from the AI model evaluator, and make the web server scale with more requests or possibly act asynchronously
  2. Rewrite the image submission processing so it waits a specified number of seconds to bunch multiple requests together into a multi-threaded queue for processing. Make sure it takes advantage of the model's runtime batch input submission capabilities.
lsha0730 commented 1 month ago

Claimed by fredgido