Open fangyiyu opened 10 months ago
Hi @fangyiyu, indeed in the new version we utilize peft library for the LoRA training and removed our own tools like convert_gpt2_attention_to_lora
but unfortunately synthetic text generation folder is still based on our previous version where we used our own LoRA implementation. At some point I will spend the time to update this folder but I cannot anticipate when. In the meantime I think if you could simply remove this part of the code where we load the model with our own LoRA implementation and just use peft library's way of loading the model with LoRA, then I think the rest should work just fine.
I finetuned GPT-2 with LORA using the latest
dp-transformers
library and 1 GPU. I am now trying to use the fine-tuned model to generate synthetic private text data using the following command-line code script following the instruction on this page:However, an error occurred:
ImportError: cannot import name 'convert_gpt2_attention_to_lora' from 'dp_transformers.module_modification'
I checked the source code ofdp_transformers.module_modification
and found that there is indeed no function calledconvert_gpt2_attention_to_lora
. Will the library be updated to make sure the synthetic private text generation could work?Thanks for your time.