Closed 2003pro closed 2 months ago
BTW, I met a similar issue on Adam version:
IsADirectoryError: [Errno 21] Is a directory: '../grads/llama2-7b-p0.05-lora-seed3/flan_v2-ckpt422-adam'
You should be using all_orig.pt
, which is the normalized gradients! Sorry for the confusion.
@2003pro Hi have you successfully run the code?
Following the script provided in the second step of "Selecting data for a task" in your readme, I have a command line that needs to be run as shown below:
However, after running it, I encountered the following error:
This location appears to be a file, but upon inspection, there are three files in that folder named all_orig.pt, all_unormalized.pt, and grads-9.pt. I wonder if I should be loading grads-9.pt at this point. Your assistance would be greatly appreciated.