Closed jqueguiner closed 2 years ago
oups, I guess we are all colliding here here is the related change 3months ago that removed the // operator because of torch working. https://github.com/huggingface/transformers/pull/13013
my 2 cents is that implicit operator are kind of broken
I feel like https://github.com/StevenTang1998 push toward explicit operation https://github.com/huggingface/transformers/pull/13013#issuecomment-894198148 but not supported for torch < 1.8
@sgugger @nreimers @patrickvonplaten
any thoughs on that matter ?
I think we will need to do a version check and use @StevenTang1998 solution for recent versions of PyTorch and leave the old code for older versions.
Do you want to take a stab at a PR?
I could try to write it if you want.
I think I just asked you to ;-)
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.
Please note that issues that do not follow the contributing guidelines are likely to be ignored.
I am still getting the error in generation_utils.py. The issue is not yet resolved in Transformers 4.15.0
@JeetRoy97,
Could you also post your environment info?
Just run transformes-cli env
and copy paste the output here. Especially your Python and PyTorch versions would be important to know
I run into the same division error torch: 1.6.0 transformers: 4.15.0 onnxruntime: 1.10.0 python: 3.6 GPU
tokenizer =M2M100Tokenizer.from_pretrained( path ) tokenizer.src_lang ='xx' example =['xxxx'] encoded = tokenizer(example, return_tensors='pt') generated_tokens = model.generate(**encoded, forced_bos_token_id=tokenizer.get_lang_id('en'))
Error occurred at the model.generate
step.
error message was directed to "transformers/generation_utils.py"
line 1955: next_indices = (next_tokens / vocab_size).long()
I am not entirely sure if you can reproduce because I used the onnx version of the M2M model.
This one looks like a similar problem: https://discuss.pytorch.org/t/runtimeerror-integer-division-of-tensors-using-div-or-is-no-longer-supported-and-in-a-future-release-div-will-perform-true-division-as-in-python-3-use-true-divide-or-floor-divide-in-python-instead/99427 and it was solved by changing the input type image = image.float()
, but in the case of NLP transformer, the inputs are ids. I tried to use encoded = encoded.to(torch.long), no luck.
Any advice would be highly appreciated!
Same error when running the fine-tuning script, run_summarization.py
with --predict_with_generate
. The stack trace:
Traceback (most recent call last):
File "run_summarization.py", line 698, in <module>
main()
File "run_summarization.py", line 650, in main
predict_results = trainer.predict(
File "/usr/local/lib/python3.8/dist-packages/transformers/trainer_seq2seq.py", line 119, in predict
return super().predict(test_dataset, ignore_keys=ignore_keys, metric_key_prefix=metric_key_prefix)
File "/usr/local/lib/python3.8/dist-packages/transformers/trainer.py", line 2319, in predict
output = eval_loop(
File "/usr/local/lib/python3.8/dist-packages/transformers/trainer.py", line 2419, in evaluation_loop
loss, logits, labels = self.prediction_step(model, inputs, prediction_loss_only, ignore_keys=ignore_keys)
File "/usr/local/lib/python3.8/dist-packages/transformers/trainer_seq2seq.py", line 172, in prediction_step
generated_tokens = self.model.generate(
File "/usr/local/lib/python3.8/dist-packages/torch/autograd/grad_mode.py", line 15, in decorate_context
return func(*args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/transformers/generation_utils.py", line 1239, in generate
return self.beam_search(
File "/usr/local/lib/python3.8/dist-packages/transformers/generation_utils.py", line 2027, in beam_search
next_indices = (next_tokens / vocab_size).long()
RuntimeError: Integer division of tensors using div or / is no longer supported, and in a future release div will perform true division as in Python 3. Use true_divide or floor_divide (// in Python) instead.
Environment:
- `transformers` version: 4.17.0.dev0
- Platform: Linux-5.4.0-90-generic-x86_64-with-glibc2.29
- Python version: 3.8.10
- PyTorch version (GPU?): 1.6.0 (False)
- Tensorflow version (GPU?): 2.3.1 (False)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: NO
- Using distributed or parallel set-up in script?: NO
Hey @mwojnars,
Thanks for reporting the error here. I'm attaching a PR that should fix it.
@patrickvonplaten Thanks. It seems to be working fine now.
The error seems to persist when running the RAG-Token model: https://huggingface.co/facebook/rag-token-nq. Running the exact code from the aforementioned link with a fresh install of the transformers
library yields the following error:
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
/tmp/ipykernel_35233/3517742745.py in <module>
1 input_dict = tokenizer.prepare_seq2seq_batch("who holds the record in 100m freestyle", return_tensors="pt")
2
----> 3 generated = model.generate(input_ids=input_dict["input_ids"])
4 print(tokenizer.batch_decode(generated, skip_special_tokens=True)[0])
5
~/anaconda3/lib/python3.8/site-packages/torch/autograd/grad_mode.py in decorate_context(*args, **kwargs)
13 def decorate_context(*args, **kwargs):
14 with self:
---> 15 return func(*args, **kwargs)
16 return decorate_context
17
~/anaconda3/lib/python3.8/site-packages/transformers/models/rag/modeling_rag.py in generate(self, input_ids, attention_mask, context_input_ids, context_attention_mask, doc_scores, max_length, min_length, early_stopping, use_cache, num_beams, num_beam_groups, diversity_penalty, bos_token_id, pad_token_id, eos_token_id, length_penalty, no_repeat_ngram_size, encoder_no_repeat_ngram_size, repetition_penalty, bad_words_ids, num_return_sequences, decoder_start_token_id, n_docs, prefix_allowed_tokens_fn, forced_bos_token_id, forced_eos_token_id, remove_invalid_values, **model_kwargs)
1614 num_beam_hyps_to_keep=num_return_sequences,
1615 )
-> 1616 return self.beam_search(
1617 input_ids,
1618 beam_scorer,
~/anaconda3/lib/python3.8/site-packages/transformers/generation_utils.py in beam_search(self, input_ids, beam_scorer, logits_processor, stopping_criteria, max_length, pad_token_id, eos_token_id, output_attentions, output_hidden_states, output_scores, return_dict_in_generate, synced_gpus, **model_kwargs)
1853 )
1854
-> 1855 next_indices = (next_tokens / vocab_size).long()
1856 next_tokens = next_tokens % vocab_size
1857
RuntimeError: Integer division of tensors using div or / is no longer supported, and in a future release div will perform true division as in Python 3. Use true_divide or floor_divide (// in Python) instead.
and the following warnings:
/home/myname/anaconda3/lib/python3.8/site-packages/transformers/models/rag/tokenization_rag.py:92: FutureWarning: `prepare_seq2seq_batch` is deprecated and will be removed in version 5 of 🤗 Transformers. Use the regular `__call__` method to prepare your inputs and the tokenizer under the `with_target_tokenizer` context manager to prepare your targets. See the documentation of your specific tokenizer for more details
warnings.warn(
/home/myname/anaconda3/lib/python3.8/site-packages/transformers/generation_utils.py:1747: UserWarning: `max_length` is deprecated in this function, use `stopping_criteria=StoppingCriteriaList(MaxLengthCriteria(max_length=max_length))` instead.
warnings.warn(
I cannot rule out that this is an error specific to my installation. But as I've just installed the transformers
library, I suspect that this might be a more general issue.
Environment info
transformers
version: 4.12.0Who can help
Models:
Information
Model I am using T5conditionalGeneration through Questgen.ai
The problem arises when using:
def predict(sentence): tokenizer = AutoTokenizer.from_pretrained("flexudy/t5-base-multi-sentence-doctor")
To reproduce
Steps to reproduce the behavior:
1.
2.
Expected behavior
No error
Fix found
https://github.com/huggingface/transformers/blob/master/src/transformers/generation_utils.py#L1839
use // instead of /