search

4AI / LS-LLaMA

A Simple but Powerful SOTA NER Model | Official Code For Label Supervised LLaMA Finetuning
https://arxiv.org/abs/2310.01208
MIT License
131 stars 20 forks source link

I'm stuck in a place, but I'm a beginner and I don't understand why, can you help me #1

Closed ckw-6 closed 11 months ago

ckw-6 commented 11 months ago

python unllama_seq_clf.py sst2 7b max Loading checkpoint shards: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:20<00:00, 10.21s/it] Some weights of UnmaskingLlamaForSequenceClassification were not initialized from the model checkpoint at NousResearch/Llama-2-7b-hf and are newly initialized: ['score.weight'] You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference. trainable params: 6,307,840 || all params: 6,613,651,456 || trainable%: 0.09537605726527117 Map: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 67349/67349 [00:01<00:00, 45129.89 examples/s] Map: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 872/872 [00:00<00:00, 18604.56 examples/s] Map: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1821/1821 [00:00<00:00, 38537.91 examples/s] 0%| | 0/168380 [00:00<?, ?it/s]You're using a LlamaTokenizerFast tokenizer. Please note that with a fast tokenizer, using the __call__ method is faster than using a method to encode the text followed by a call to the pad method to get a padded encoding. The progress bar here is always 0.

ckw-6 commented 11 months ago

@SeanLee97 If you are free sometime, would you please help me, thank you very much

SeanLee97 commented 11 months ago

@ckw-6 It works fine on my machine. You'd better run a small dataset like SST5 to make sure the environment is OK.

image

csroyli commented 11 months ago

I think the program is running well, as the model has been downloaded and loaded to your machine. Probably it is caused by the large model, making the training so slow. Suggest check your GPU utility and see if the GPU is loaded. Btw, what is your GPU? I ran on RTX4090, and it works fine.

ckw-6 commented 11 months ago

Thank you very much for your help. I found that there was a problem with my cuda, and the problem has been solved. Thanks again for your warm help。