10up / classifai

Supercharge WordPress Content Workflows and Engagement with Artificial Intelligence.
https://classifaiplugin.com
GNU General Public License v2.0
569 stars 53 forks source link

Integrate Microsoft AI/ML for image classification #51

Closed jeffpaul closed 5 years ago

jeffpaul commented 5 years ago

Splitting this out from #28, via @jeffpaul:

add in support for Microsoft AI/ML to propvide image descriptions

Initial integration (aka MVP):

jeffpaul commented 5 years ago

Will want to ensure this includes UI for filtering in the Media Library (currently in backbone.js).

rickalee commented 5 years ago

Note from our initial research into Azure Cognitive Services Computer Vision API, I would suggest that we add option for default tolerance of 'confidence' score of returned caption to ensure we get accurate descriptions. Based on preliminary tests, we have found captions need to be over 80-85% confidence.

https://azure.microsoft.com/en-us/services/cognitive-services/computer-vision/

We should also add option to populate description field and/or alt text.

jeffpaul commented 5 years ago

@rickalee thanks for the input, and that matches our findings in building out support for IBM Watson (the confidence level bit) and definitely planning to apply to description/alt text. Any other uses of the Computer Vision API that you can think of from your research?

jeffpaul commented 5 years ago

@helen note the updated details in this issue related to MVP and post-MVP requirements, hoping to allow us to get something released in support of Azure AI and then come back in a subsequent release to add more media support.

helen commented 5 years ago

@rickalee I ended up going with 75 as the default/recommended minimum after testing some of the different kinds of images I might upload to a blog post. That said, I'm very open to tweaking that if you've found that it's a serious issue as a default value, there was definitely more fuzz in there in what I got as results but still better than no alt text by default, at least for general usage.