Open Jasonzzt opened 5 months ago
@leonardozcm pls take a look, whether it is not supported by our kernel? tks.
hi, I think the VF.drop
is not implemented by our kernels, instead I suppose this error indicates that input
is in 8-bit data format which is not a supported dtype for torch.nn.functional.dropout
@Jasonzzt From the log, it is found that PPO also applies PEFT LoRA.
Therefore, like QLoRA, rather than from_pretrained
a peft model with lora config, we should first load the base model, and then use get_peft_model
, prepare_model_for_kbit_training
etc. methods in qlora.py to create a peft model. Such a model is built on top of layers with supported operators like here.
Machine: MAX1100 ipex-llm: 2.1.0b20240421 bigdl-core-xe-21 2.5.0b20240421 bigdl-core-xe-esimd-21 2.5.0b20240421 Related PR When trying to run trl PPO finetuning on MAX1100, I got the following error.