Added PaliGemma notebook examples for zero-shot object detection and instance segmentation.

google-gemini / gemma-cookbook

A collection of guides and examples for the Gemma open models from Google.

https://ai.google.dev/gemma/

Apache License 2.0

668 stars 123 forks source link

Added PaliGemma notebook examples for zero-shot object detection and instance segmentation. #11

Closed NSTiwari closed 4 months ago

NSTiwari commented 4 months ago

Added notebook examples for PaliGemma for zero-shot object detection and instance segmentation on images and videos. Please review, thanks. :)

jethac commented 4 months ago

Thanks for this, Nitin! I'll take a look now at what you've done.

jethac commented 4 months ago

Hey Nitin, thanks again for your work. Unfortunately the notes in CONTRIBUTING.md envisage you actually uploading your notebooks into this repository, not just linking out to yours - apologies if this wasn't clear.

As to why this is required, it's such that we can make sure everything that Gemma users encounter in the cookbook is known by us to run - if we allow a link out to a notebook somewhere else, we have no visibility into any of that.

If you'd be okay with it, I'd encourage you to please:

add sections to your notebooks for how to get Kaggle/HF/etc tokens and set them as Colab secrets
add a byline section to your notebooks such that anyone reading them knows that you are the original author (and knows where to find you!)
resubmit this PR with your notebooks in it

Best,

Jetha

NSTiwari commented 4 months ago

Hi @jethac, I've just made the changes suggested by you.

Added a section to my notebooks to configure HF token in Colab Secrets.
Added a byline section about me at the top of the notebooks.

Could you please review?

windmaple commented 4 months ago

Haven't looked at the code in detail, but for the 2 notebooks involving videos, is there any way to add a sample video?

NSTiwari commented 4 months ago

Haven't looked at the code in detail, but for the 2 notebooks involving videos, is there any way to add a sample video?

@windmaple: Do you want me to add the original video I used for the demo or any video would do? Also, where do I add these videos? Any specific folder within the repo?

windmaple commented 4 months ago

@NSTiwari could you create an 'assets' subfolder in PaliGemma folder and add the video you have? Assuming the vid is not too big (say, a few MB?)

NSTiwari commented 4 months ago

@NSTiwari could you create an 'assets' subfolder in PaliGemma folder and add the video you have? Assuming the vid is not too big (say, a few MB?)

@windmaple: I just added a couple of videos as sample. Could you please check?

windmaple commented 4 months ago

Turns out the videos are too big unfortunately (each >25MB). Let's leave them out for now. Sorry for going back and forth on this.

I'll merge once you remove the vids.

NSTiwari commented 4 months ago

Turns out the videos are too big unfortunately (each >25MB). Let's leave them out for now. Sorry for going back and forth on this.

I'll merge once you remove the vids.

Thanks, @windmaple. I just deleted videos. Please check.