search

OpenGVLab / Ask-Anything

[CVPR2024 Highlight][VideoChatGPT] ChatGPT with video understanding! And many more supported LMs such as miniGPT4, StableLM, and MOSS.
https://vchat.opengvlab.com/
MIT License
2.85k stars 230 forks source link

Any instructions for fine-tuning on custom datasets? #155

Open 2000ZRL opened 3 months ago

2000ZRL commented 3 months ago

Thanks for your excellent work. I am curious if there are any instructions for fine-tuning video-llava on my own dataset?

yinanhe commented 3 months ago

You can do so by adding your meta file in the instruction_data file. The meta file is a json and follows the following schema:

[
  {
    "video": "string",
    "QA": {
      "i": "string",
      "q": "string",
      "a": "string"
    }
  },
  {
    "image": "string",
    "QA": {
      "i": "string",
      "q": "string",
      "a": "string"
    }
  },
  ...
]