Cannot import name 'convert_gpt2_attention_to_lora' from 'dp_transformers.module_modification'

[fangyiyu]

I finetuned GPT-2 with LORA using the latest dp-transformers library and 1 GPU. I am now trying to use the fine-tuned model to generate synthetic private text data using the following command-line code script following the instruction on this page:

python3 generate-text.py \
    --model_type gpt2 \
    --model_name_or_path output \
    --input_training_file data \
    --output_dir synthetic_data \
    --length 128 \
    --total_sequences 100 \
    --do_sample \
    --batch_size 8 \
    --lora_dim 4 \
    --lora_alpha 32 \
    --lora_dropout`` 0.0 \

However, an error occurred: ImportError: cannot import name 'convert_gpt2_attention_to_lora' from 'dp_transformers.module_modification' I checked the source code of dp_transformers.module_modification and found that there is indeed no function called convert_gpt2_attention_to_lora. Will the library be updated to make sure the synthetic private text generation could work?

Thanks for your time.

[huseyinatahaninan]

Hi @fangyiyu, indeed in the new version we utilize peft library for the LoRA training and removed our own tools like convert_gpt2_attention_to_lora but unfortunately synthetic text generation folder is still based on our previous version where we used our own LoRA implementation. At some point I will spend the time to update this folder but I cannot anticipate when. In the meantime I think if you could simply remove this part of the code where we load the model with our own LoRA implementation and just use peft library's way of loading the model with LoRA, then I think the rest should work just fine.