Open daiyizheng opened 9 months ago
/slurm/home/yrd/shaolab/daiyizheng/.conda/envs/llama/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: /slurm/home/yrd/shaolab/daiyizheng/.conda/envs/llama did not contain libcudart.so as expected! Searching further paths...
warn(msg)
/slurm/home/yrd/shaolab/daiyizheng/.conda/envs/llama/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('/run/user/80104/vscode-git-4b808d81bf.sock')}
warn(msg)
/slurm/home/yrd/shaolab/daiyizheng/.conda/envs/llama/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('/run/user/80104/vscode-ipc-0b328df5-e364-494a-b230-9f7e99271b5b.sock')}
warn(msg)
/slurm/home/yrd/shaolab/daiyizheng/.conda/envs/llama/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('() { eval /usr/bin/modulecmd bash $*
\n}')}
warn(msg)
The tokenizer class you load from this checkpoint is not the same type as the class this function is called from. It may result in unexpected tokenization.
The tokenizer class you load from this checkpoint is 'LLaMATokenizer'.
The class this function is called from is 'LlamaTokenizer'.
The model weights are not tied. Please use the tie_weights
method before using the infer_auto_device
function.
Loading checkpoint shards: 0%| | 0/81 [00:00<?, ?it/s]
Loading checkpoint shards: 1%| | 1/81 [00:00<01:08, 1.16it/s]
Loading checkpoint shards: 2%|▏ | 2/81 [00:01<01:11, 1.11it/s]
Loading checkpoint shards: 4%|▎ | 3/81 [00:02<01:13, 1.07it/s]
Loading checkpoint shards: 5%|▍ | 4/81 [00:03<01:10, 1.09it/s]
Loading checkpoint shards: 6%|▌ | 5/81 [00:04<01:09, 1.09it/s]
Loading checkpoint shards: 7%|▋ | 6/81 [00:05<01:09, 1.08it/s]
Loading checkpoint shards: 9%|▊ | 7/81 [00:06<01:10, 1.05it/s]
Loading checkpoint shards: 10%|▉ | 8/81 [00:07<01:07, 1.08it/s]
Loading checkpoint shards: 11%|█ | 9/81 [00:08<01:08, 1.05it/s]
Loading checkpoint shards: 12%|█▏ | 10/81 [00:09<01:06, 1.07it/s]
Loading checkpoint shards: 14%|█▎ | 11/81 [00:10<01:05, 1.06it/s]
Loading checkpoint shards: 15%|█▍ | 12/81 [00:11<01:04, 1.08it/s]
Loading checkpoint shards: 16%|█▌ | 13/81 [00:12<01:02, 1.08it/s]
Loading checkpoint shards: 17%|█▋ | 14/81 [00:13<01:02, 1.07it/s]
Loading checkpoint shards: 19%|█▊ | 15/81 [00:14<01:03, 1.05it/s]
Loading checkpoint shards: 20%|█▉ | 16/81 [00:15<01:02, 1.04it/s]
Loading checkpoint shards: 21%|██ | 17/81 [00:16<01:02, 1.02it/s]
Loading checkpoint shards: 22%|██▏ | 18/81 [00:16<01:01, 1.03it/s]
Loading checkpoint shards: 23%|██▎ | 19/81 [00:17<01:00, 1.03it/s]
Loading checkpoint shards: 25%|██▍ | 20/81 [00:18<00:59, 1.02it/s]
Loading checkpoint shards: 26%|██▌ | 21/81 [00:19<00:58, 1.02it/s]
Loading checkpoint shards: 27%|██▋ | 22/81 [00:20<00:57, 1.02it/s]
Loading checkpoint shards: 28%|██▊ | 23/81 [00:21<00:57, 1.01it/s]
Loading checkpoint shards: 30%|██▉ | 24/81 [00:22<00:55, 1.04it/s]
Loading checkpoint shards: 31%|███ | 25/81 [00:23<00:52, 1.07it/s]
Loading checkpoint shards: 32%|███▏ | 26/81 [00:24<00:51, 1.07it/s]
Loading checkpoint shards: 33%|███▎ | 27/81 [00:25<00:50, 1.07it/s]
Loading checkpoint shards: 35%|███▍ | 28/81 [00:26<00:49, 1.06it/s]
Loading checkpoint shards: 36%|███▌ | 29/81 [00:27<00:48, 1.08it/s]
Loading checkpoint shards: 37%|███▋ | 30/81 [00:28<00:47, 1.07it/s]
Loading checkpoint shards: 38%|███▊ | 31/81 [00:29<00:45, 1.09it/s]
Loading checkpoint shards: 40%|███▉ | 32/81 [00:30<00:45, 1.07it/s]
Loading checkpoint shards: 41%|████ | 33/81 [00:31<00:44, 1.09it/s]
Loading checkpoint shards: 42%|████▏ | 34/81 [00:31<00:42, 1.10it/s]
Loading checkpoint shards: 43%|████▎ | 35/81 [00:32<00:42, 1.09it/s]
Loading checkpoint shards: 44%|████▍ | 36/81 [00:33<00:40, 1.10it/s]
Loading checkpoint shards: 46%|████▌ | 37/81 [00:34<00:38, 1.13it/s]
Loading checkpoint shards: 47%|████▋ | 38/81 [00:35<00:38, 1.12it/s]
Loading checkpoint shards: 48%|████▊ | 39/81 [00:36<00:37, 1.12it/s]
Loading checkpoint shards: 49%|████▉ | 40/81 [00:37<00:37, 1.08it/s]
Loading checkpoint shards: 51%|█████ | 41/81 [00:38<00:36, 1.11it/s]
Loading checkpoint shards: 52%|█████▏ | 42/81 [00:39<00:34, 1.11it/s]
Loading checkpoint shards: 53%|█████▎ | 43/81 [00:40<00:34, 1.10it/s]
Loading checkpoint shards: 54%|█████▍ | 44/81 [00:41<00:34, 1.08it/s]
Loading checkpoint shards: 56%|█████▌ | 45/81 [00:42<00:33, 1.08it/s]
Loading checkpoint shards: 57%|█████▋ | 46/81 [00:43<00:33, 1.06it/s]
Loading checkpoint shards: 58%|█████▊ | 47/81 [00:43<00:31, 1.07it/s]
Loading checkpoint shards: 59%|█████▉ | 48/81 [00:44<00:30, 1.09it/s]
Loading checkpoint shards: 60%|██████ | 49/81 [00:45<00:29, 1.08it/s]
Loading checkpoint shards: 62%|██████▏ | 50/81 [00:46<00:29, 1.07it/s]
Loading checkpoint shards: 63%|██████▎ | 51/81 [00:47<00:27, 1.07it/s]
Loading checkpoint shards: 64%|██████▍ | 52/81 [00:48<00:27, 1.05it/s]
Loading checkpoint shards: 65%|██████▌ | 53/81 [00:49<00:27, 1.03it/s]
Loading checkpoint shards: 67%|██████▋ | 54/81 [00:50<00:25, 1.04it/s]
Loading checkpoint shards: 68%|██████▊ | 55/81 [00:51<00:26, 1.00s/it]
Loading checkpoint shards: 69%|██████▉ | 56/81 [00:52<00:24, 1.02it/s]
Loading checkpoint shards: 70%|███████ | 57/81 [00:53<00:23, 1.02it/s]
Loading checkpoint shards: 72%|███████▏ | 58/81 [00:54<00:22, 1.04it/s]
Loading checkpoint shards: 73%|███████▎ | 59/81 [00:55<00:20, 1.06it/s]
Loading checkpoint shards: 74%|███████▍ | 60/81 [00:56<00:19, 1.06it/s]
Loading checkpoint shards: 75%|███████▌ | 61/81 [00:57<00:19, 1.05it/s]
Loading checkpoint shards: 77%|███████▋ | 62/81 [00:58<00:18, 1.05it/s]
Loading checkpoint shards: 78%|███████▊ | 63/81 [00:59<00:17, 1.04it/s]
Loading checkpoint shards: 79%|███████▉ | 64/81 [01:00<00:15, 1.06it/s]
Loading checkpoint shards: 80%|████████ | 65/81 [01:01<00:14, 1.07it/s]
Loading checkpoint shards: 81%|████████▏ | 66/81 [01:01<00:13, 1.08it/s]
Loading checkpoint shards: 83%|████████▎ | 67/81 [01:02<00:12, 1.10it/s]
Loading checkpoint shards: 84%|████████▍ | 68/81 [01:03<00:12, 1.08it/s]
Loading checkpoint shards: 85%|████████▌ | 69/81 [01:04<00:10, 1.10it/s]
Loading checkpoint shards: 86%|████████▋ | 70/81 [01:05<00:10, 1.09it/s]
Loading checkpoint shards: 88%|████████▊ | 71/81 [01:06<00:09, 1.07it/s]
Loading checkpoint shards: 89%|████████▉ | 72/81 [01:07<00:08, 1.09it/s]
Loading checkpoint shards: 90%|█████████ | 73/81 [01:08<00:07, 1.05it/s]
Loading checkpoint shards: 91%|█████████▏| 74/81 [01:09<00:06, 1.07it/s]
Loading checkpoint shards: 93%|█████████▎| 75/81 [01:10<00:05, 1.08it/s]
Loading checkpoint shards: 94%|█████████▍| 76/81 [01:11<00:04, 1.09it/s]
Loading checkpoint shards: 95%|█████████▌| 77/81 [01:12<00:03, 1.08it/s]
Loading checkpoint shards: 96%|█████████▋| 78/81 [01:13<00:02, 1.05it/s]
Loading checkpoint shards: 98%|█████████▊| 79/81 [01:14<00:01, 1.06it/s]
Loading checkpoint shards: 99%|█████████▉| 80/81 [01:15<00:00, 1.06it/s]
Loading checkpoint shards: 100%|██████████| 81/81 [01:15<00:00, 1.21it/s]
Loading checkpoint shards: 100%|██████████| 81/81 [01:15<00:00, 1.07it/s]
Traceback (most recent call last):
File "/slurm/home/yrd/shaolab/daiyizheng/nlp/Huatuo-Llama-Med-Chinese/infer.py", line 132, in
我不规范的解决方案:
我使用llama 65b 在多GPU A100 infer.py推理的时候报了错误, 在finetune.py没有问题