I am facing an issue when attempting to run the 'run_generation.py' script from the 'unlimiformer/src' directory. This script is crucial for my work with Unlimiformer on Google Colab, and I need assistance in resolving the problem.
When executing the following command:
python src/run_generation.py --model_type llama --model_name_or_path meta-llama/Llama-2-13b-chat-hf \
--prefix "<s>[INST] <<SYS>>\n You are a helpful assistant. Answer with detailed responses according to the entire instruction or question. \n<</SYS>>\n\n Summarize the following book: " \
--prompt example_inputs/harry_potter_full.txt \
--suffix " [/INST]" --test_unlimiformer --fp16 --length 200 --layer_begin 16 \
--index_devices 1 --datastore_device 1
the script runs on Google Colab, but it abruptly terminates with the following error message:
10/05/2023 11:16:12 - WARNING - __main__ - device: cuda, n_gpu: 1, 16-bits training: True
Using pad_token, but it is not set yet.
^C
The script utilizes the 'cuda' device, and I have one GPU ('n_gpu: 1') available for the process.
I am enabling 16-bits training with the '--fp16' flag.
The script also specifies various parameters, including 'length,' 'layer_begin,' 'index_devices,' and 'datastore_device.'
The issue arises when running the script but does not provide a clear indication of the problem's root cause.
I am facing an issue when attempting to run the 'run_generation.py' script from the 'unlimiformer/src' directory. This script is crucial for my work with Unlimiformer on Google Colab, and I need assistance in resolving the problem.
When executing the following command:
The script utilizes the 'cuda' device, and I have one GPU ('n_gpu: 1') available for the process. I am enabling 16-bits training with the '--fp16' flag. The script also specifies various parameters, including 'length,' 'layer_begin,' 'index_devices,' and 'datastore_device.' The issue arises when running the script but does not provide a clear indication of the problem's root cause.