Implement OpenWhisk integration [on the server side] for Alchemy Vision APIs.

IBM / BluePic

WARNING: This repository is no longer maintained :warning: This repository will not be updated. The repository will be kept available in read-only mode.

https://developer.ibm.com/code/patterns

Apache License 2.0

325 stars 67 forks source link

Implement OpenWhisk integration [on the server side] for Alchemy Vision APIs. #14

Closed rolivieri closed 8 years ago

rolivieri commented 8 years ago

The inputs are: 1) image/picture (NSData) and 2) the image record ID for the corresponding record in the Cloudant database.
The AlchemyVision API is then used to process the image to perform object and scene recognition.
The output from the AlchemyVision is a list that contains one or more classifications along with the confidence level (e.g. "dog", 99%). This output should be stored in the corresponding image JSON document in Cloudant under the tags field.
There's a new method in the RouteUtils.swift file (invokeOpenWhisk()) that will server as the starting point for kicking off the OpenWhisk sequence.

triceam commented 8 years ago

Shouldn't be a problem as long as Swift on Linux/Whisk is up to the task. It will basically be grabbing the image data, and submitting that the Alchemy/Watson vision APIs. We may need to do some image processing (resize) for the analysis. The state of image APIs in Swift on Linux is unknown until we dig deeper.

Are there specific requirements for this use case yet?

There is a JavaScript implementation that does something similar here: https://github.com/IBM-Bluemix/openwhisk-darkvisionapp/blob/master/processing/analysis.js

triceam commented 8 years ago

If we need to resize images, perhaps we can sequence or wrap the existing actions available here: https://github.ibm.com/BlueMix-Fabric/bluewhisk/blob/master/blue/catalog/actions/thumbnail/thumbnail.js

rolivieri commented 8 years ago

@triceam Added a few details above to address your question on requirements. Also, see the comments I added to issue #29 on JS actions. Look forward to hearing your thoughts on this.

triceam commented 8 years ago

Looks like the demo for Alchemy supports passing in the fully qualified url to an attachment in Cloudant. It might be possible to just pass the url in our implementation so we don't have to deal with image binaries. I'm hoping this works for Object storage too, but time will tell... Ex: https://username:password@4690f31b-c2a1-4d73-a90a-560ce540122d-bluemix.cloudant.com/overwatch/031E930D-283E-4B96-81F1-0D867E97757A/thumbnail.jpg

rolivieri commented 8 years ago

@triceam - We are no longer storing images in Cloudant; instead, they are now stored in Object Storage. The images in Object Storage are accessible using a URL (we configure the container in Object Storage to make this possible).

triceam commented 8 years ago

@rolivieri: OK, thanks. Can you share the structure of a saved document in Cloudant that is ready for analysis? I just need to see what the JSON looks like. We can probably grab this off of the Cloudant web interface, but I don't have the full environment setup locally.

rolivieri commented 8 years ago

@triceam - Yes, we have the structure of image documents already defined. You can find sample image documents in the following file: BluePic-Server/resources/cloudant/images.json. As of now, the URL for the image (in the Object Storage repository) is not stored in the JSON document. We have planned to store it in the JSON document but haven't gotten to it. If having the URL in the JSON document helps you, let us know so we can address this change soon.

rolivieri commented 8 years ago

Moving all OpenWhisk issues/work items to milestore 0.4.

triceam commented 8 years ago

implemented sequence. see https://github.com/IBM-Swift/BluePic/pull/206