Have the parameters of llm's input embedding been tuned?

[LB0828]

Thank you for providing the code. In the paper, the introduction of new tokens, marked as <\path>, is mentioned. I have a question regarding the tuning of input embeddings for the language model (llm) parameters. I noticed in the training code, specifically within the get_input_embeddings().parameters(), the requires_grad property is not explicitly set to true. Could you please clarify the necessity for this tuning?

[RManLuo]

We do not use the LoRA in our experiments. If I understand the package of trl and huggingface Trainer correctly, all the parameters of the model will be set as trainable by calling model.train().