Open PaulWang0513 opened 11 months ago
Hello Authors,
Encountered the above RuntimeError related to tensor size mismatch while trying to train the sparsity predictor as per the instructions in the README. The issue arises when running ./run_infer_opt_175b_collect_sp_data.sh. All relevant files are created successfully until the script hits a tensor size error.
RuntimeError: The expanded size of the tensor (2048) must match the existing size (2047) at non-singleton dimension 1. Target sizes: [1, 2048]. Tensor sizes: [2047]
Can you please provide additional information on how can we resolve the issue ?
Thank You .
I met the same error when tp=1. Looking for the author to fix it. A tricky way to solve it is using tp=2. I have verified it and the error disappears.
Hi everyone! I am facing this issue right now, after trying to use facebook/opt-125m
model instead. Anyone found a fix?
@zhaoyang-star Where is this parameter tp=2
?
Same things happens to me too when I'm trying to use facebook/opt-1.7b
model in colab notebook. Waiting for any possible solution.
I met the same error when tp=1. Looking for the author to fix it. A tricky way to solve it is using tp=2. I have verified it and the error disappears.
@zhaoyang-star Is tp
a library or a parameter? Does it mean the top-p
in c4_train/getdata.py
? Thanks for your help!
@2455DD I reached out to the author earlier this month, and she replied with this "I believe there is a temporary work around suggested on GitHub by setting the top_p = 2 in get_data file."
So, you're correct, it is top-p
in c4_train/getdata.py
I followed the instructions and modified the file DejaVu/Decentralized_FM_alpha/c4_train/getdata.py in the following way:
data = { "best_of": 1, "echo": True, "logprobs": 1, "max_tokens": 0, "model": "opt-1.3b", "n": 1, "prompt": doc["text"], "request_type": "language-model-inference", "stop": None, "temperature": 0, "top_p": 2, } After making these changes, I deleted the local Hugging Face cache of C4 and the file c4_train.jsonl. I then re-executed getdata.py and ran the script run_infer_opt_175b_collect_sp_data.sh. However, I am still encountering a size mismatch error: xxx RuntimeError: The size of tensor a (2047) must match the size of tensor b (2048) at non-singleton dimension 3
@2455DD I reached out to the author earlier this month, and she replied with this "I believe there is a temporary work around suggested on GitHub by setting the top_p = 2 in get_data file."
So, you're correct, it is
top-p
inc4_train/getdata.py
Same things happens to me too when I'm trying to use
facebook/opt-1.7b
model in colab notebook. Waiting for any possible solution.I met the same error when tp=1. Looking for the author to fix it. A tricky way to solve it is using tp=2. I have verified it and the error disappears.
@zhaoyang-star Is
tp
a library or a parameter? Does it mean thetop-p
inc4_train/getdata.py
? Thanks for your help!
I used the run_infer_opt_1_3_b_collect_sp_data.sh script from your forked repository, but I encountered the same size mismatch error. Did you run the script successfully?
@2455DD I reached out to the author earlier this month, and she replied with this "I believe there is a temporary work around suggested on GitHub by setting the top_p = 2 in get_data file." So, you're correct, it is
top-p
inc4_train/getdata.py
Same things happens to me too when I'm trying to use
facebook/opt-1.7b
model in colab notebook. Waiting for any possible solution.I met the same error when tp=1. Looking for the author to fix it. A tricky way to solve it is using tp=2. I have verified it and the error disappears.
@zhaoyang-star Is
tp
a library or a parameter? Does it mean thetop-p
inc4_train/getdata.py
? Thanks for your help!I used the run_infer_opt_1_3_b_collect_sp_data.sh script from your forked repository, but I encountered the same size mismatch error. Did you run the script successfully?
Can confirm I am having a similar issue after setting "model": "opt-125m", "top_p":2
. Only difference is I get
RuntimeError: The expanded size of the tensor (2048) must match the existing size (2047) at non-singleton dimension 1. Target sizes: [1, 2048]. Tensor sizes: [2047]
(the difference being that the mismatch happens at dimension 1 and not 3).
Hi, I was trying to train the sparsity predictor by following the instructions in the README file, but encountered some error when running
./run_infer_opt_175b_collect_sp_data.sh
.Following is the printed message:
I used WSL2 with Ubuntu 20.04 and CUDA 11.3. Due to storage limitations, I changed some setting to use a smaller pretrained model (opt-125m). In
DejaVu/Decentralized_FM_alpha/c4_train/get_data.py
:In
DejaVu/Decentralized_FM_alpha/convert_opt_checkpoint.py
:In
DejaVu/Decentralized_FM_alpha/run_infer_opt_175b_collect_sp_data.sh
:The att_sp_x_0.mmap ~ att_sp_x_11.mmap, mlp_sp_x_0.mmap ~ mlp_sp_x_11.mmap, mlp_label_0.mmap ~ mlp_label_11.mmap, and score_norm_0 ~ score_norm_11.mmap are created successfully before the error occured. Can you think of any possible reason or solution? Thanks!