search

WoodWing / elvis-image-recognition

Elvis integration with AI based image recognition services from AWS, Google and Clarifai
MIT License
11 stars 10 forks source link

Set recognition model and vendor by metadata #13

Open sitescripter opened 6 years ago

sitescripter commented 6 years ago

Currently the selection of vendor and model is defined by assetDomain and assetPath.

Case 1: Photographer uploads images with category "Celebrity". We would like to use the info and send the images to the corresponding model.

Case 2: Files archived from Enterprise contain an editorial category. We might want to use sceCategory for setting the model

Case 3: To reduce cost it would be convenient if we could avoid sending agency images to recognition - !usageRights:agency

All cases can be solved by the ability to map queries to model and vendor.

jaapvanblaaderen commented 6 years ago

Interesting idea... but could be a bit tricky to implement. Question is how far you want to go with the queries? Identical to everything that's possible in Elvis?

The tricky part is that the image recognition integration doesn't have knowledge of the Elvis query language (think about interpreting something like: "category:Celebrity AND (status:final OR licenseStartDate:[2018-05-01 TO 2018-12-31])". This is something Elasticsearch (Lucene) can interpret inside Elvis. I now see two options if we want to implement queries like these in this integration:

Option A: use the Elvis search API to perform a queries on the given asset. Downside of this option is that you'd have to perform separate queries for every mapping you have. This might put a heavy API load on Elvis.

Option B: Create logic in the integration to interpret the query and make the selection locally. This would be very nice, but is probably really complex to implement (you'd have to re-invent Lucene query parsing and searching...).

Both options are far from ideal. An another alternative might be to support less features in the query, for example a subset of the options you'd have in Lucene or just a single metadata=value match. Ideally I would opt for something where the integration can match the right model without too much additional logic and without extra API calls towards Elvis.

sitescripter commented 6 years ago

Keeping this integration focused on recognition is key. Perhaps combine two webhooks? The first will react on asset_create and other events, have the logic required to set the model (model field per vendor?), and flag the record for being picked up using a boolean field cf_aiRecognize. A sample for doing this would be welcome. The second webhook could then react if cf_aiRecognize is changed to true and use the model set by the first webhook to call this integration.