search

OpenGVLab / ChartAst

ChartAssistant is a chart-based vision-language model for universal chart comprehension and reasoning.
Other
64 stars 5 forks source link

Questions about the training details about ChartAst-S #7

Closed zengxingchen closed 4 months ago

zengxingchen commented 5 months ago

Thanks for your excellent work! After carefully reading the paper, I still have questions about the trainable setting of ChartAst-S. Which part of the model is trainable/frozen in pertaining and instruction tuning stages, respectively? Vision tower, projection layer, and LLM?

Thx again.

FanqingM commented 5 months ago

Same with SPHINX, We train the projection, part of vision encoders and part of LLM (llama2)