search

AIDC-AI / Ovis

A novel Multimodal Large Language Model (MLLM) architecture, designed to structurally align visual and textual embeddings.
https://huggingface.co/AIDC-AI/Ovis1.6-Gemma2-9B
Apache License 2.0
506 stars 29 forks source link

How to finetune the model？ #16

Closed cyj95 closed 1 month ago

cyj95 commented 2 months ago

any script for fully finetuning and lora finetuning?

runninglsy commented 1 month ago

You can modify the parameters ovis_pretrained_path and dataset_names in the following script to finetune Ovis1.5: https://github.com/AIDC-AI/Ovis/blob/main/scripts/v1_5/Ovis1.5-Llama3-8B-S3.sh. Specifically, ovis_pretrained_path should be set to the local directory of Ovis 1.5 checkpoints (which can be downloaded from https://huggingface.co/AIDC-AI/Ovis1.5-Llama3-8B/tree/main).