Queue and fetching is much slower than it should be for a simple JSON and 512x512 image. Maybe we should have a socket instead of HTTP requests? Maybe compression is necessary (at least putting the image in webp form or something)? For me it takes longer to send and receive the data than it actually does for my machine to run inference.
Queue and fetching is much slower than it should be for a simple JSON and 512x512 image. Maybe we should have a socket instead of HTTP requests? Maybe compression is necessary (at least putting the image in webp form or something)? For me it takes longer to send and receive the data than it actually does for my machine to run inference.