search

facebookresearch / access

Code to reproduce the experiments from the paper.
Other
101 stars 36 forks source link

The evaluate.py script fails; pytorch version issue #20

Closed taziksh closed 4 years ago

taziksh commented 4 years ago

See the following stack trace upon running python scripts/evaluate.py:

Traceback (most recent call last):
  File "scripts/evaluate.py", line 27, in <module>
    print(evaluate_simplifier_on_turkcorpus(simplifier, phase='test'))
  File "/content/access/access/evaluation/general.py", line 25, in evaluate_simplifier_on_turkcorpus
    pred_filepath = get_prediction_on_turkcorpus(simplifier, phase)
  File "/content/access/access/evaluation/general.py", line 20, in get_prediction_on_turkcorpus
    simplifier(source_filepath, pred_filepath)
  File "/content/access/access/simplifiers.py", line 32, in wrapped
    simplifier(complex_filepath, pred_filepath)
  File "/content/access/access/simplifiers.py", line 66, in preprocessed_simplifier
    simplifier(preprocessed_complex_filepath, preprocessed_output_pred_filepath)
  File "/content/access/access/simplifiers.py", line 32, in wrapped
    simplifier(complex_filepath, pred_filepath)
  File "/content/access/access/simplifiers.py", line 46, in fairseq_simplifier
    fairseq_generate(complex_filepath, output_pred_filepath, exp_dir, **kwargs)
  File "/content/access/access/fairseq/base.py", line 283, in fairseq_generate
    batch_size=batch_size)
  File "/content/access/access/fairseq/base.py", line 238, in _fairseq_generate
    generate.main(generate_args)
  File "/usr/local/lib/python3.6/dist-packages/fairseq_cli/generate.py", line 106, in main
    hypos = task.inference_step(generator, models, sample, prefix_tokens)
  File "/usr/local/lib/python3.6/dist-packages/fairseq/tasks/fairseq_task.py", line 242, in inference_step
    return generator.generate(models, sample, prefix_tokens=prefix_tokens)
  File "/usr/local/lib/python3.6/dist-packages/torch/autograd/grad_mode.py", line 15, in decorate_context
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/fairseq/sequence_generator.py", line 378, in generate
    scores.view(bsz, beam_size, -1)[:, :, :step],
  File "/usr/local/lib/python3.6/dist-packages/fairseq/search.py", line 83, in step
    torch.div(self.indices_buf, vocab_size, out=self.beams_buf)
RuntimeError: Integer division of tensors using div or / is no longer supported, and in a future release div will perform true division as in Python 3. Use true_divide or floor_divide (// in Python) instead.

To reproduce, run python scripts/evaluate.py

chuang0310 commented 4 years ago

See the following stack trace upon running python scripts/evaluate.py:

Traceback (most recent call last):
  File "scripts/evaluate.py", line 27, in <module>
    print(evaluate_simplifier_on_turkcorpus(simplifier, phase='test'))
  File "/content/access/access/evaluation/general.py", line 25, in evaluate_simplifier_on_turkcorpus
    pred_filepath = get_prediction_on_turkcorpus(simplifier, phase)
  File "/content/access/access/evaluation/general.py", line 20, in get_prediction_on_turkcorpus
    simplifier(source_filepath, pred_filepath)
  File "/content/access/access/simplifiers.py", line 32, in wrapped
    simplifier(complex_filepath, pred_filepath)
  File "/content/access/access/simplifiers.py", line 66, in preprocessed_simplifier
    simplifier(preprocessed_complex_filepath, preprocessed_output_pred_filepath)
  File "/content/access/access/simplifiers.py", line 32, in wrapped
    simplifier(complex_filepath, pred_filepath)
  File "/content/access/access/simplifiers.py", line 46, in fairseq_simplifier
    fairseq_generate(complex_filepath, output_pred_filepath, exp_dir, **kwargs)
  File "/content/access/access/fairseq/base.py", line 283, in fairseq_generate
    batch_size=batch_size)
  File "/content/access/access/fairseq/base.py", line 238, in _fairseq_generate
    generate.main(generate_args)
  File "/usr/local/lib/python3.6/dist-packages/fairseq_cli/generate.py", line 106, in main
    hypos = task.inference_step(generator, models, sample, prefix_tokens)
  File "/usr/local/lib/python3.6/dist-packages/fairseq/tasks/fairseq_task.py", line 242, in inference_step
    return generator.generate(models, sample, prefix_tokens=prefix_tokens)
  File "/usr/local/lib/python3.6/dist-packages/torch/autograd/grad_mode.py", line 15, in decorate_context
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/fairseq/sequence_generator.py", line 378, in generate
    scores.view(bsz, beam_size, -1)[:, :, :step],
  File "/usr/local/lib/python3.6/dist-packages/fairseq/search.py", line 83, in step
    torch.div(self.indices_buf, vocab_size, out=self.beams_buf)
RuntimeError: Integer division of tensors using div or / is no longer supported, and in a future release div will perform true division as in Python 3. Use true_divide or floor_divide (// in Python) instead.

To reproduce, run python scripts/evaluate.py

You could try to use torch==1.2

louismartin commented 4 years ago

Yes I guess you need to downgrade your pytorch version :(

taziksh commented 4 years ago

I got it to work! Also had to downgrade sacrebleu to match torch's version.

!pip install torch==1.2
!pip install sacrebleu==1.3.7

Do you want me to make a PR to update the README.md with this info?

louismartin commented 4 years ago

Hi thanks for proposing, yes you can update the README :) The torch version is included in requirements.txt.