search

JoshuaChou2018 / SkinGPT-4

Pre-trained Multimodal Large Language Model Enhances Dermatological Diagnosis using SkinGPT-4
44 stars 4 forks source link

Querry regarding further improvement #4

Closed sidbnoida closed 4 months ago

sidbnoida commented 5 months ago

Hi.. I am a dermatologist and have read your work with interest. I had one querry. In your work you have fine-tuned the alignment layer of the model in two steps over the step 1 pre-trained checkpoint provided in the mini gpt-4 code. Is it also possible to fine-tune the vicuna 7b model with dermatology specific dataset to make a custom dermatology vicuna model which could answer the questions better. Also if we plan to do that, would it mean that we would need to do the step 1 training of the mini -gpt 4 model all over again or can we proceed with just the step-2 training like you did with the mini gpt-4 model?

JoshuaChou2018 commented 4 months ago

Hi, thank you for your message. Yes, you can fine-tune the vicuna 7b with your own data. In that case, it is suggested to train the alignment layer again from step 1.