微调权重加载 - Githubissues

robblecrunch commented 1 year ago

Is there an existing issue for this?

[X] I have searched the existing issues

Current Behavior

17 prefix_state_dict = torch.load(os.path.join(CHECKPOINT_PATH, "pytorch_model.bin")) 18 new_prefix_state_dict = {} 19 for k, v in prefix_state_dict.items(): 20 if k.startswith("transformer.prefix_encoder."): 21 new_prefix_state_dict[k[len("transformer.prefix_encoder."):]] = v 22 model.transformer.prefix_encoder.load_state_dict(new_prefix_state_dict)

请问，我有多个下游微调任务，可一次加载吧？如，17~22指定二个下游任务的微调权重，加载二次。

Expected Behavior

No response

Steps To Reproduce

分析微调了二个下游任务
推理时能一次加载吗

Environment

- OS: centos
- Python: 3.9
- Transformers: 4.30
- PyTorch: 2.0.1+cu118
- CUDA Support (`python -c "import torch; print(torch.cuda.is_available())"`) : True

Anything else?

No response

zmeet-ai commented 1 year ago

遇到微调后模型污染主模型的问题了吗？

Lxr713 commented 1 year ago