search

Shared-Reality-Lab / IMAGE-server

IMAGE project server components
Other
2 stars 7 forks source link

IMAGE not working with AVIF graphics #679

Open VenissaCarolQuadros opened 1 year ago

VenissaCarolQuadros commented 1 year ago

IMAGE does not work for AVIF graphics. I tried with Photo 1 and Photo 2

An error of this kind is logged at the various priority groups (logs from unicorn): FetchError: invalid json response body at http://collage-detector-preprocessor:5000/preprocessor reason: Unexpected token '<', "<!doctype "... is not valid JSON

This website has a jpg, WebP and avif version of the same photo (and IMAGE works for the first 2), which might serve as a nice test case. Photo 2 is taken from this repo; A person is detected in the jpg equivalent.

JRegimbal commented 1 year ago

After looking through several containers, it seems that AVIF graphics are not supported by opencv which is used in nearly all our preprocessors. It is likely that it or the preprocessors are not failing gracefully when given an AVIF graphic. Is this format particularly common on the internet now?

jeffbl commented 1 year ago

Moving away from opencv sounds messy. An alternative way of dealing with this would be for the orchestrator to convert it to jpg or another usable format before passing it along.

JRegimbal commented 1 year ago

I don't particularly like the orchestrator doing extra tasks on the fly like this. Putting aside anything conceptual, it needs to run with more permissions than other components and I do not think we should be making a habit of adding functionality running with those permissions. Maybe restricting the kinds of graphic types supported as inputs so non-supported types such as AVIF are rejected?

jeffbl commented 1 year ago

Looks like AVIF is used by fewer than 1% of websites: https://w3techs.com/technologies/details/im-avif

Leaving in backlog. Should be fixed, but agree needs more discussion.

jeffbl commented 1 year ago

Looks like AVIF is now supported by OpenCV, so this might just mean updating our OpenCV version to v4.8.0 or later https://github.com/opencv/opencv/issues/19271 https://opencv.org/blog/2023/07/02/opencv-4-8-0/