search

OpenNMT / OpenNMT-py

Open Source Neural Machine Translation and (Large) Language Models in PyTorch
https://opennmt.net/
MIT License
6.73k stars 2.25k forks source link

Error message of `SequenceTooLongError` #2522

Closed PC91 closed 9 months ago

PC91 commented 10 months ago

Hello,

I got the following error when the input data is too long.

Traceback (most recent call last):
  File "/opt/conda/lib/python3.10/site-packages/onmt/trainer.py", line 503, in _gradient_accumulation
    model_out, attns = self.model(
  File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/opt/conda/lib/python3.10/site-packages/onmt/models/model.py", line 189, in forward
    enc_out, enc_final_hs, src_len = self.encoder(src, src_len)
  File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/opt/conda/lib/python3.10/site-packages/onmt/encoders/transformer.py", line 207, in forward
    enc_out = self.embeddings(src)
  File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/opt/conda/lib/python3.10/site-packages/onmt/modules/embeddings.py", line 291, in forward
    source = module(source, step=step)
  File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/opt/conda/lib/python3.10/site-packages/onmt/modules/embeddings.py", line 72, in forward
    raise SequenceTooLongError(
onmt.modules.embeddings.SequenceTooLongError: Sequence is 6438 but PositionalEncoding is limited to 1. See max_len argument.
Traceback (most recent call last):
  File "/opt/conda/bin/onmt_train", line 8, in <module>
    sys.exit(main())
  File "/opt/conda/lib/python3.10/site-packages/onmt/bin/train.py", line 67, in main
    train(opt)
  File "/opt/conda/lib/python3.10/site-packages/onmt/bin/train.py", line 49, in train
    p.join()
  File "/opt/conda/lib/python3.10/multiprocessing/process.py", line 149, in join
    res = self._popen.wait(timeout)
  File "/opt/conda/lib/python3.10/multiprocessing/popen_fork.py", line 43, in wait
    return self.poll(os.WNOHANG if timeout == 0.0 else 0)
  File "/opt/conda/lib/python3.10/multiprocessing/popen_fork.py", line 27, in poll
    pid, sts = os.waitpid(self.pid, flag)
  File "/opt/conda/lib/python3.10/site-packages/onmt/utils/distributed.py", line 159, in signal_handler
    raise Exception(msg)
Exception:

When I had a look at this line, it seems that the logged limit of PositionalEncoding is not correctly shown. I think pe.size(1) should be logged (instead of self.pe.size(1)) since the array was transposed. Could you confirm that it needs to be updated?

Thank you!

vince62s commented 10 months ago

this is exoected https://github.com/OpenNMT/OpenNMT-py/blob/master/onmt/modules/embeddings.py#L27 don't use traditional positional encoding if you have long sequences.