Feature Request: Implement Parallel processing for image file Parsing

[Kickercost]

Currently detection can only handle a single image at a time. This causes Multi camera systems with multiple image generations per second to have their record events delayed until irrelevant.

I believe the heart of this issue is tied to the following line in the OnCreatedAsync event handler. while (detection_running == true) { } //wait until other detection process is finished

Unfortunately this requires a refactor of how file handling tasks are handled in general and the whole handler would need to be refactored to allow for async task creation, execution and followup execution

[gentlepumpkin]

the limiting factor is Deepstack, if you send multiple requests to deepstack in parallel it will still answer them one after the other.

[Kickercost]

Well that is definitely unfortunate. I wonder if there is any way to get around that :/

[doudar]

I think you could use kubernetes to spawn multiple instances of deepstack, but I've been playing around with it and cannot even get two nodes connected yet. My plan would be to run deepstack wherever there is idle hardware in the house. (small handful of desktop computers that are each more powerful than my BI machine and only get used a few hours a day).

[SHerms]

I just asked for this feature in one of the active pull requests. https://github.com/gentlepumpkin/bi-aidetection/pull/32#issuecomment-682071868

@VorlonCD was nice enough to do a first pass.

I have yet to try it out. https://github.com/gentlepumpkin/bi-aidetection/pull/32/commits/8bdf62319369de4f9217dacb87cca55d02aaa7e4

[Kickercost]

I've taken a look at the underlying python used in deep stack and it indeed is only spawning a single threaded handler for the image processing.

Similar to what Doudar suggested:

I think you could use kubernetes to spawn multiple instances of deepstack, but I've been playing around with it and cannot even get two nodes connected yet. My plan would be to run deepstack wherever there is idle hardware in the house. (small handful of desktop computers that are each more powerful than my BI machine and only get used a few hours a day).

I have spawned multiple instances of the underlying python scripts running detection as well as parallel executions of the AI tool with distinct directories service wrappers for each. This works but actually doesnt entirely fix the delayed processing issue. As it turns out, reading the file from disk is what results in most of the delay experienced by the loop in my test. So with that in mind I'm proposing the following changes as it relates to this feature:

1. Since, loading images from the disk is the one of slowest parts of the processing loop. If we could instead continue loading future images to be processed from disk into a queue to be processed that would speed iterations up.

   • We will still need to honor the timeout periods of inactivity in the detection and throw away any images that were loaded within the window.

2. Multiple Camera systems would benefit from having parallel processing (assuming deepstack multithreading is fixed\worked around)

   • If applying the multiple Deep Stack Instance running then each camera would need to allow overriding the api url for deep stack or the api url itself would need to be moved to the camera object itself at the very least
   • Else if the Deep Stack Python scripts are tweaked for multiple run handler spawns then a single deep stack URL could still be used.
   • Each camera would need its own processing thread or the image processing loop itself would need to be made async and its logic compartmentalized in such a way to allow multiple tread safe executions.
   • Each camera would benefit from having its own processing queue of images to parse with a centralized file reader process that would be responsible for assigning newly loaded images to their matching cameras queue. (ties into point 1)

Thoughts on this proposal? I haven't had a lot of time the last few months but with the holidays coming up Ill have a bit more freetime and wouldn't mind taking a crack at it. Hopefully my rambling makes sense.

(I just read the pull #32 request so I realize some of my statements might be irrelevant with the work already performed, ill look more later)

[VorlonCD]

@Kickercost - With my version you can specify multiple deepstack URLs (url1:port ; url2:port ; etc) and it will also queue up the images to wait in line to be processed as needed.

So you could have the Windows version of deepstack installed, a Docker version of deepstack running on another port, and MAYBE even an older/different docker version running in Docker (Like the GPU version or a different beta version).

And of course as many other deepstack installs as you want on other machines, including Raspberry PI's's's to do even more concurrent processing. Of course if multiple are running on the same machine it would hammer the CPU or GPU, but they would still run concurrently because they are totally separate instances.

https://github.com/VorlonCD/bi-aidetection