search

starmpcc / Asclepius

Official Codes for "Publicly Shareable Clinical Large Language Model Built on Synthetic Clinical Notes"
89 stars 6 forks source link

A problem with Asclepius-R-7B: TypeError: The current model class (LlamaModel) is not compatible with `.generate()`, as it doesn't have a language model head. Please use one of the following classes instead: {'LlamaForCausalLM'} #4

Closed scherbakovdmitri closed 9 months ago

scherbakovdmitri commented 9 months ago

Hello all, I have followed installation requirements outlines in the model page by setting up conda, python, and downloading model checkpoints. I get this error:

tokenizer = AutoTokenizer.from_pretrained('.', use_fast=False) You are using the default legacy behaviour of the <class 'transformers.models.llama.tokenization_llama.LlamaTokenizer'>. If you see this, DO NOT PANIC! This is expected, and simply means that the legacy (previous) behavior will be used so nothing changes for you. If you want to use the new behaviour, set legacy=True. This should only be set if you understand what it means, and thouroughly read the reason why this was added as explained in https://github.com/huggingface/transformers/pull/24565 model = AutoModel.from_pretrained('.') /opt/conda/envs/asclepius/lib/python3.9/site-packages/transformers/utils/generic.py:260: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead. torch.utils._pytree._register_pytree_node( /opt/conda/envs/asclepius/lib/python3.9/site-packages/transformers/utils/generic.py:260: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead. torch.utils._pytree._register_pytree_node( Loading checkpoint shards: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:20<00:00, 6.75s/it]

note = "This is a sample note" question = "What is the diagnosis?"

model_input = prompt.format(note=note, question=question) input_ids = tokenizer(model_input, return_tensors="pt").input_ids output = model.generate(input_ids) Traceback (most recent call last): File "", line 1, in File "/opt/conda/envs/asclepius/lib/python3.9/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(*args, **kwargs) File "/opt/conda/envs/asclepius/lib/python3.9/site-packages/transformers/generation/utils.py", line 1402, in generate self._validate_model_class() File "/opt/conda/envs/asclepius/lib/python3.9/site-packages/transformers/generation/utils.py", line 1197, in _validate_model_class raise TypeError(exception_message) TypeError: The current model class (LlamaModel) is not compatible with .generate(), as it doesn't have a language model head. Please use one of the following classes instead: {'LlamaForCausalLM'}

starmpcc commented 9 months ago

Sorry, this is a same error as https://github.com/starmpcc/Asclepius/commit/927eb0c7bac75b801175b1164c750a320335b6d5

Please modify the below two lines: from transformers import AutoTokenizer, AutoModel to from transformers import AutoTokenizer, AutoModelForCausalLM model = AutoModel.from_pretrained({DOWNLOADED_PATH}) to model = AutoModelForCausalLM.from_pretrained({DOWNLOADED_PATH}).

We would update the instruction immediately.

Thank you!