Closed fungiboletus closed 1 year ago
I have reproduced your error. It seems like with dummy weight, the model will generate very random output_ids
like the following:
In [25]: output_ids[3:4, 500:]
Out[25]:
array([[ 1, 1, 1,
1, 1, 2,
32826, 16, 5,
812, 343, 9,
72340172838076672, 0, 0, 0, 0, 1,
1, 1, 1,
1, 1, 1,
1, 1, 1,
1, 1, 1,
1, 1, 1,
1, 1, 1,
1, 1, 1,
1, 1, 1,
1, 1]])
A hot fix is included in #50.
Thanks! I ran the benchmark and got 12.71 tokens using two Tesla A30.
Hi,
I'm trying to run the benchmark
bench_30b_1x4.sh
(except that I setN_GPUS=2
), but I get the following python exception:I use Python
3.10.9
with Pytorch1.13.1
with Cuda11.7
, and mpirun2.1.1
.