search

chaoyi-wu / RadFM

The official code for "Towards Generalist Foundation Model for Radiology by Leveraging Web-scale 2D&3D Medical Data".
315 stars 32 forks source link

lang_encoder_path #12

Open zzzzxciid opened 10 months ago

zzzzxciid commented 10 months ago

I am in the process of training a model and I have a question regarding the lang_encoder_path. I've placed the pytorch_model.bin file in it. However, I encountered an issue where I received the following message: "Some weights of the model checkpoint at ./Language_files were not used when initializing LlamaForCausalLM."

Could you provide guidance on this matter?

Hamesen commented 3 months ago

I'm training a model using train.py, but I'm running into a problem. Where can I download the models in the lang_encoder_path and tokenizer_path below? If not using the one you provided, is it possible to replace it with a generic model?

@chaoyi-wu @zzzzxciid class ModelArguments: lang_encoder_path: Optional[str] = field(default="/home/cs/leijiayu/wuchaoyi/book_pretrain/Results/Book_mix_2048_13B_full/checkpoint-45800") tokenizer_path: str = field(default='/home/cs/leijiayu/wuchaoyi/Finetune_LLAMA/LLAMA_Model/tokenizer', metadata={"help": "Path to the tokenizer data."})