API interrogation tags overlap between requests

[WhiteMemory99]

Recently I migrated from Toriato's implementation to yours and faced some issues with parallel interrogation requests via API. Whenever there's multiple images in the queue, your extension tends to merge the tags that belong to different images into a single response for each request. Toriato's version handles every API request properly.

Tested on webui 1.5.1 and 1.6.0-RC

[picobyte]

I've been working a bit on a feature branch, I suspect what may cause the issue for you is the QData usage that is still in use in the master branch. This branch was intended also to improve the queuing/interrogation, but I'd like it to be fairly debugged before I push it in main. I have synced it with the master branch.

My initial response mentioned that the averaging was intended but that is not the case for API. In this feature branch it is possible to interrogate with different interrogation models and average that, but it's per image.

Still I'd like to have a test case, otherwise this api will remain hard to debug..

[picobyte]

There were some fixes required, but now the aforementioned feature branch works, using below in the console, I don't see the tag blending here, will try my master branch. or maybe it's just that I don't understand what the hack you were doing. In the future, if you send an error report, at least mention that.

curl -X 'POST' \
  'http://127.0.0.1:7860/tagger/v1/interrogate' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{ "image": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNk+A8AAQUBAScY42YAAAAASUVORK5CYII=", "model": "wd14-vit.v1", "threshold": 0.35 }'

curl -X 'POST' \
  'http://127.0.0.1:7860/tagger/v1/interrogate' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{ "image": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAgAAAAIAQMAAAD+wSzIAAAABlBMVEX///+/v7+jQ3Y5AAAADklEQVQI12P4AIX8EAgALgAD/aNpbtEAAAAASUVORK5CYII=",  "model": "wd14-vit.v1", "threshold": 0.35 }'

[picobyte]

On the master branch I tried reproducing your issue, curling first the first, the second and then the first again. I cannot reproduce the issue that you seem to have.

curl -X 'POST'   'http://127.0.0.1:7860/tagger/v1/interrogate'   -H 'accept: application/json'   -H 'Content-Type: application/json'   -d '{ "image": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAgAAAAIAQMAAAD+wSzIAAAABlBMVEX///+/v7+jQ3Y5AAAADklEQVQI12P4AIX8EAgALgAD/aNpbtEAAAAASUVORK5CYII=",  "model": "wd14-vit.v1", "threshold": 0.35 }' |md5sum
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  3699  100  3498  100   201   3552    204 --:--:-- --:--:-- --:--:--  3755
be7cab9b685a6c1ece1114f40ca2c4f3  -

curl -X 'POST'   'http://127.0.0.1:7860/tagger/v1/interrogate'   -H 'accept: application/json'   -H 'Content-Type: application/json'   -d '{ "image": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNk+A8AAQUBAScY42YAAAAASUVORK5CYII=", "model": "wd14-vit.v1", "threshold": 0.35 }' | md5sum
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  3699  100  3527  100   172   3618    176 --:--:-- --:--:-- --:--:--  3793
fae3b39319162ebd0ce857649448e2db  -

curl -X 'POST'   'http://127.0.0.1:7860/tagger/v1/interrogate'   -H 'accept: application/json'   -H 'Content-Type: application/json'   -d '{ "image": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAgAAAAIAQMAAAD+wSzIAAAABlBMVEX///+/v7+jQ3Y5AAAADklEQVQI12P4AIX8EAgALgAD/aNpbtEAAAAASUVORK5CYII=",  "model": "wd14-vit.v1", "threshold": 0.35 }' |md5sum
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  3699  100  3498  100   201   3597    206 --:--:-- --:--:-- --:--:--  3801
be7cab9b685a6c1ece1114f40ca2c4f3  -

[picobyte]

My idea was to add an async batch runner that would keep going until a queue endpoint is issued, but this requirement to also account for the possibility that multiple users may be batch querying at the same server - if that is the issue you are having - is going to make this a lot more complicated. Now there also needs to be a queue name introduced, and then there may be multiple different interrogators being requested by multiple users at the same time and the response handling is even more tough.

[picobyte]

Notes concerning async handling in the fastapi (WIP).

The general idea is that you browse the net to collect, prepare an image and send it to the interrogator, which interrogates in the background while you continue browsing. Since you browse during interrogation time, at the end of the day you have to wait less for the interrogation.

Possible schema:

{
  "image": "",
  "model": "string",
  "threshold": 0.35,
  "queue": "string",
  "name": "string"
}

If queue is empty, interrogate normally, synchronous; name is ignored.

If queue is set (it should be unique and for your particular queue), the image is not necessarily immediately interrogated, but added to a queue. All subsequent images for this queue should use the same model (at least for now). (currently queue is a string, but could also be an increment, also maybe a get_queue_id() would be the right way)

name the name to queue the image with (just use the image filename) when empty, the queue is marked finished. A result is awaited.

There is a runner that for each model being queried, interrogates the images, FIFO model, FIFO images per model. when a queue gets empty, a next model is loaded, and so on, possibly repeating (if images were not queued fast enough) until all images are done.

[picobyte]

Changes were applied with #64