Hi!
I'm trying to reproduce the sparsity result on OPT-30b model. Threre are 48 transformer layers in OPT-30B. I've convert the weight from HF format to pt format and got 48 pytorch_.pt files. However, in the sparse_predictor/main_mlp.py, the CONFIG["30b"]["num_layer"] is 24, not 48. I am confused what does the num_layer and the rest args mean here? Should I modify these args or not? @lzcemma
Thanks.
Hi! I'm trying to reproduce the sparsity result on OPT-30b model. Threre are 48 transformer layers in OPT-30B. I've convert the weight from HF format to pt format and got 48 pytorch_.pt files. However, in the sparse_predictor/main_mlp.py, the CONFIG["30b"]["num_layer"] is 24, not 48. I am confused what does the num_layer and the rest args mean here? Should I modify these args or not? @lzcemma Thanks.