search

ariG23498 / ft-pali-gemma

Notebooks for fine tuning pali gemma
MIT License
41 stars 1 forks source link

Train paligemma with additional info #3

Open DigitalPathology opened 2 weeks ago

DigitalPathology commented 2 weeks ago

I have this dataset format:

{"file_name": "1.png", "question": "detect mitosis", "multiple_choice_answer": "<loc0425><loc0421><loc0549><loc0539> mitosis"}

I would like to train paligemma with addtional info. For example:

"info": "Scanned by Leica"

How can I introduce this additional info to paligemma?

ariG23498 commented 1 week ago

You can take a look at the creating dataset script for a better understanding of how this process would be done in order to fine tune PaliGemma.