search

goberoi / cloudy_vision

Cloudy Vision is an open source tool to test the image labeling capabilities of different computer vision API vendors.
MIT License
230 stars 64 forks source link

Add several metrics, statistics, rekognition vendor and ability to work with tagged images #2

Closed lucasdchamps closed 7 years ago

lucasdchamps commented 7 years ago

Hi Gaurav,

I'm Lucas, working for Recast.ai. We recently had to compare several image recognition APIs to pick the best one for us. First, thank you very much for providing cloudy_vision, it worked perfectly and saved us a lot of time.

While performing the benchmark, we enhanced cloudy_vision with several features we would like to share with you:

That's a lot of changes for a single PR but they have been split into atomic commits to ease the review ;)

The additional metrics and statistics improved the benchmark relevance and we hope to see them integrated into cloudy_vision!

goberoi commented 7 years ago

Wow, that's awesome! Thank you for the nice words, and even more so for giving back.

I will have a chance to review this next week, apologies for my slowness, but I hope to include it soon.

Thanks again, and best of luck with Recast.ai!

Gaurav

On Wed, Jan 11, 2017 at 2:02 PM, lucasdchamps notifications@github.com wrote:

Hi Gaurav,

I'm Lucas, working for Recast.ai https://recast.ai. We recently had to compare several image recognition APIs to pick the best one for us. First, thank you very much for providing cloudy_vision, it worked perfectly and saved us a lot of time.

While performing the benchmark, we enhanced cloudy_vision with several features we would like to share with you:

  • response_time and tags_count metrics
  • ability to compute vendor statistics (average and standard deviation)
  • rekognition vendor
  • ability to work with tagged images

That's a lot of changes for a single PR but they have been split into atomic commits to ease the review ;)

The additional metrics and statistics improved the benchmark relevance and we hope to see them integrated into cloudy_vision!

You can view, comment on, or merge this pull request online at:

https://github.com/goberoi/cloudy_vision/pull/2 Commit Summary

  • [ADD] venv to .gitignore.
  • [ADD] resize setting, only import cv2 and resize when setting is set to True.
  • [UPD] call_vision_api returns json instead of text, review vendors dependencies.
  • [ADD] response_time metrics
  • [ADD] vendor statistics.
  • [ADD] tags_count metric.
  • [ADD] rekognition vendor.
  • [ADD] Ability to work with tagged images.

File Changes

Patch Links:

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/goberoi/cloudy_vision/pull/2, or mute the thread https://github.com/notifications/unsubscribe-auth/AAB4CUJ9PgHY3GPEmnMAfGjbV3dMuPUpks5rRVFmgaJpZM4LhH8o .

lucasdchamps commented 7 years ago

Hi, did you have time to check it out? Thanks.

goberoi commented 7 years ago

Hi @lucasdchamps, apologies for my slowness, I finally had a chance to review and accept your changes today.

Really nice work. I appreciate the new features: tagging, latency measurement, and Amazon's API. I also like your attention to detail on small things like: making resizing and cv2 import an option, and fixing my kludge on returning text instead of Python structures from each vendor call.

I plan on updating the blog post and static generated content over the next couple of days, and I'll be sure to point to your contributions. Thanks again!