search

matanr / capex

CAPE using text-graphs
Apache License 2.0
7 stars 1 forks source link

What prompts we can use for vehicle other parts, like door, window , hood, bumper etc #3

Open Chaitanya887 opened 2 weeks ago

Chaitanya887 commented 2 weeks ago

@matanr All the predefined keypoints are working fine, but can we use to detect car hotspots (different parts of the vehicles)

matanr commented 2 weeks ago

Hi @Chaitanya887, Indeed the system was designed to work on the predefined text descriptions. Sometimes it works on other prompts. See the experiments (Section 4) and Figures 5-8 in the paper. However, as discussed in the limitations (Section 5) and shown in Figure 9, our model was trained on relatively small dataset over a short period of time, which limits its open-vocabulary ability.

I would suggest to try different text variations and play with the model, but unfortunatley there is a chance your desired unique text description won't work.

Chaitanya887 commented 2 weeks ago

Thank you @matanr, for your response. Is there any way to find-tune the base model / train the model for only a specific category let's say vehicles, so that it would able to predict maximum keypoint for different parts of the vehicle with pretty good accuracy ? If yes, what dataset should be used, and how it should be formatted, to train it ?