search

yonatanbitton / wysiwyr

Apache License 2.0
30 stars 0 forks source link

PaLI Checkpoint or Training Script #9

Open nicholscrawford opened 11 months ago

nicholscrawford commented 11 months ago

Is there a way to replicate the PaLI results? Either a training script, or ideally a checkpoint would be awesome. I'm hoping to use it as a component in a research project.

yonatanbitton commented 11 months ago

Thanks for your interest. Sadly we won't be able to share the PaLI checkpoint as this is internal model. However, we do plan to publish the training dataset soon, and this way you can reproduce the VNLI training with BLIP2 or any more updated VLM available.

Updated the BLIP2 fine-tuning instructions here: https://github.com/yonatanbitton/wysiwyr/blob/main/README.md#reproducing-results-for-the-end-to-end-vnli-method

yonatanbitton commented 9 months ago

Hi, Hi, the dataset is updated in the project website, sorry for the delay, please ask if anything is unclear CSV: https://seetrue.s3.amazonaws.com/wysiwyr_train.csv Images: https://drive.google.com/file/d/1M1CKmYkIdpFYjCOc9JwXHP5Z7E91CJl3/view?usp=drive_link