[GroundingDino] - GroundingDinoProcessor kwargs is Broken

[EduardoPach]

System Info

While working on https://github.com/huggingface/transformers/pull/31828 I realized that GroundingDinoProcessor.__call__ passes the kwargs only to the self.tokenizer which is not ideal specially if one wants to pass annotations to self.image_processor as it's currently not possible.

It should be an easy fix following https://github.com/huggingface/transformers/pull/31197

Who can help?

@amyeroberts

Information

• [ ] The official example scripts
• [ ] My own modified scripts

Tasks

• [ ] An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• [ ] My own task or dataset (give details below)

Reproduction

from transformers import AutoProcessor
from datasets import load_dataset

processor = AutoPocessor.from_pretrained("IDEA-Research/grounding-dino-tiny")
ds = load_dataset("EduardoPacheco/aquarium-sample", split="train")
id2label = {0: "fish", 1: "jellyfish", 2: "penguins", 3: "sharks", 4: "puffins", 5: "stingrays", 6: "starfish"}
prompt = ". ".join(id2label.values()) + "."

# Passes `annotations` as kwargs to tokenizer instead of image_processor
inputs = processor(images=ds["image"], text=[prompt, prompt], annotations=ds["annotations"], return_tensors="pt")

Expected behavior

Kwargs being correctly mapped to tokenizer and image processor

[NielsRogge]

We're tracking them all at https://github.com/huggingface/transformers/issues/31911, cc @zucchini-nlp I guess we can add Grounding DINO to that list

[zucchini-nlp]

Sure, added to the list. Guess we can close this issue now and track Grounding DINO in 31911 :)

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.