search

kingyiusuen / image-to-latex

Convert images of LaTex math equations into LaTex code.
MIT License
2.03k stars 311 forks source link

train error with self created data #6

Closed qiudao767 closed 3 years ago

qiudao767 commented 3 years ago 
/pytorch/aten/src/ATen/native/cuda/Indexing.cu:702: indexSelectLargeIndex: block: [37,0,0], thread: [58,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/pytorch/aten/src/ATen/native/cuda/Indexing.cu:702: indexSelectLargeIndex: block: [37,0,0], thread: [59,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/pytorch/aten/src/ATen/native/cuda/Indexing.cu:702: indexSelectLargeIndex: block: [37,0,0], thread: [60,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/pytorch/aten/src/ATen/native/cuda/Indexing.cu:702: indexSelectLargeIndex: block: [37,0,0], thread: [61,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/pytorch/aten/src/ATen/native/cuda/Indexing.cu:702: indexSelectLargeIndex: block: [37,0,0], thread: [62,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/pytorch/aten/src/ATen/native/cuda/Indexing.cu:702: indexSelectLargeIndex: block: [37,0,0], thread: [63,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/pytorch/aten/src/ATen/native/cuda/Indexing.cu:702: indexSelectLargeIndex: block: [37,0,0], thread: [63,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
Error executing job with overrides: ['trainer.gpus=1', 'data.batch_size=8']
Traceback (most recent call last):
  File "run_experiment.py", line 42, in <module>
    main()
  File "/home/nd/anaconda3/envs/img2latex/lib/python3.8/site-packages/hydra/main.py", line 49, in decorated_main
    _run_hydra(
  File "/home/nd/anaconda3/envs/img2latex/lib/python3.8/site-packages/hydra/_internal/utils.py", line 367, in _run_hydra
    run_and_report(
  File "/home/nd/anaconda3/envs/img2latex/lib/python3.8/site-packages/hydra/_internal/utils.py", line 214, in run_and_report
    raise ex
  File "/home/nd/anaconda3/envs/img2latex/lib/python3.8/site-packages/hydra/_internal/utils.py", line 211, in run_and_report
    return func()
  File "/home/nd/anaconda3/envs/img2latex/lib/python3.8/site-packages/hydra/_internal/utils.py", line 368, in <lambda>
    lambda: hydra.run(
  File "/home/nd/anaconda3/envs/img2latex/lib/python3.8/site-packages/hydra/_internal/hydra.py", line 110, in run
    _ = ret.return_value
  File "/home/nd/anaconda3/envs/img2latex/lib/python3.8/site-packages/hydra/core/utils.py", line 233, in return_value
    raise self._return_value
  File "/home/nd/anaconda3/envs/img2latex/lib/python3.8/site-packages/hydra/core/utils.py", line 160, in run_job
    ret.return_value = task_function(task_cfg)
  File "run_experiment.py", line 36, in main
    trainer.tune(lit_model, datamodule=datamodule)
  File "/home/nd/anaconda3/envs/img2latex/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 688, in tune
    result = self.tuner._tune(model, scale_batch_size_kwargs=scale_batch_size_kwargs, lr_find_kwargs=lr_find_kwargs)
  File "/home/nd/anaconda3/envs/img2latex/lib/python3.8/site-packages/pytorch_lightning/tuner/tuning.py", line 54, in _tune
    result['lr_find'] = lr_find(self.trainer, model, **lr_find_kwargs)
  File "/home/nd/anaconda3/envs/img2latex/lib/python3.8/site-packages/pytorch_lightning/tuner/lr_finder.py", line 250, in lr_find
    trainer.tuner._run(model)
  File "/home/nd/anaconda3/envs/img2latex/lib/python3.8/site-packages/pytorch_lightning/tuner/tuning.py", line 64, in _run
    self.trainer._run(*args, **kwargs)
  File "/home/nd/anaconda3/envs/img2latex/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 758, in _run
    self.dispatch()
  File "/home/nd/anaconda3/envs/img2latex/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 799, in dispatch
    self.accelerator.start_training(self)
  File "/home/nd/anaconda3/envs/img2latex/lib/python3.8/site-packages/pytorch_lightning/accelerators/accelerator.py", line 96, in start_training
    self.training_type_plugin.start_training(trainer)
  File "/home/nd/anaconda3/envs/img2latex/lib/python3.8/site-packages/pytorch_lightning/plugins/training_type/training_type_plugin.py", line 144, in start_training
    self._results = trainer.run_stage()
  File "/home/nd/anaconda3/envs/img2latex/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 809, in run_stage
    return self.run_train()
  File "/home/nd/anaconda3/envs/img2latex/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 871, in run_train
    self.train_loop.run_training_epoch()
  File "/home/nd/anaconda3/envs/img2latex/lib/python3.8/site-packages/pytorch_lightning/trainer/training_loop.py", line 499, in run_training_epoch
    batch_output = self.run_training_batch(batch, batch_idx, dataloader_idx)
  File "/home/nd/anaconda3/envs/img2latex/lib/python3.8/site-packages/pytorch_lightning/trainer/training_loop.py", line 738, in run_training_batch
    self.optimizer_step(optimizer, opt_idx, batch_idx, train_step_and_backward_closure)
  File "/home/nd/anaconda3/envs/img2latex/lib/python3.8/site-packages/pytorch_lightning/trainer/training_loop.py", line 434, in optimizer_step
    model_ref.optimizer_step(
  File "/home/nd/anaconda3/envs/img2latex/lib/python3.8/site-packages/pytorch_lightning/core/lightning.py", line 1403, in optimizer_step
    optimizer.step(closure=optimizer_closure)
  File "/home/nd/anaconda3/envs/img2latex/lib/python3.8/site-packages/pytorch_lightning/core/optimizer.py", line 214, in step
    self.__optimizer_step(*args, closure=closure, profiler_name=profiler_name, **kwargs)
  File "/home/nd/anaconda3/envs/img2latex/lib/python3.8/site-packages/pytorch_lightning/core/optimizer.py", line 134, in __optimizer_step
    trainer.accelerator.optimizer_step(optimizer, self._optimizer_idx, lambda_closure=closure, **kwargs)
  File "/home/nd/anaconda3/envs/img2latex/lib/python3.8/site-packages/pytorch_lightning/accelerators/accelerator.py", line 329, in optimizer_step
    self.run_optimizer_step(optimizer, opt_idx, lambda_closure, **kwargs)
  File "/home/nd/anaconda3/envs/img2latex/lib/python3.8/site-packages/pytorch_lightning/accelerators/accelerator.py", line 336, in run_optimizer_step
    self.training_type_plugin.optimizer_step(optimizer, lambda_closure=lambda_closure, **kwargs)
  File "/home/nd/anaconda3/envs/img2latex/lib/python3.8/site-packages/pytorch_lightning/plugins/training_type/training_type_plugin.py", line 193, in optimizer_step
    optimizer.step(closure=lambda_closure, **kwargs)
  File "/home/nd/anaconda3/envs/img2latex/lib/python3.8/site-packages/torch/optim/lr_scheduler.py", line 65, in wrapper
    return wrapped(*args, **kwargs)
  File "/home/nd/anaconda3/envs/img2latex/lib/python3.8/site-packages/torch/optim/optimizer.py", line 88, in wrapper
    return func(*args, **kwargs)
  File "/home/nd/anaconda3/envs/img2latex/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 28, in decorate_context
    return func(*args, **kwargs)
  File "/home/nd/anaconda3/envs/img2latex/lib/python3.8/site-packages/torch/optim/adamw.py", line 65, in step
    loss = closure()
  File "/home/nd/anaconda3/envs/img2latex/lib/python3.8/site-packages/pytorch_lightning/trainer/training_loop.py", line 732, in train_step_and_backward_closure
    result = self.training_step_and_backward(
  File "/home/nd/anaconda3/envs/img2latex/lib/python3.8/site-packages/pytorch_lightning/trainer/training_loop.py", line 823, in training_step_and_backward
    result = self.training_step(split_batch, batch_idx, opt_idx, hiddens)
  File "/home/nd/anaconda3/envs/img2latex/lib/python3.8/site-packages/pytorch_lightning/trainer/training_loop.py", line 290, in training_step
    training_step_output = self.trainer.accelerator.training_step(args)
  File "/home/nd/anaconda3/envs/img2latex/lib/python3.8/site-packages/pytorch_lightning/accelerators/accelerator.py", line 204, in training_step
    return self.training_type_plugin.training_step(*args)
  File "/home/nd/anaconda3/envs/img2latex/lib/python3.8/site-packages/pytorch_lightning/plugins/training_type/training_type_plugin.py", line 155, in training_step
    return self.lightning_module.training_step(*args, **kwargs)
  File "/home/nd/PycharmProjects/imagetolatex/image_to_latex/lit_models/lit_resnet_transformer.py", line 55, in training_step
    logits = self.model(imgs, targets[:, :-1])
  File "/home/nd/anaconda3/envs/img2latex/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/nd/PycharmProjects/imagetolatex/image_to_latex/models/resnet_transformer.py", line 88, in forward
    output = self.decode(y, encoded_x)  # (Sy, B, num_classes)
  File "/home/nd/PycharmProjects/imagetolatex/image_to_latex/models/resnet_transformer.py", line 122, in decode
    y = self.embedding(y) * math.sqrt(self.d_model)  # (Sy, B, E)
  File "/home/nd/anaconda3/envs/img2latex/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/nd/anaconda3/envs/img2latex/lib/python3.8/site-packages/torch/nn/modules/sparse.py", line 158, in forward
    return F.embedding(
  File "/home/nd/anaconda3/envs/img2latex/lib/python3.8/site-packages/torch/nn/functional.py", line 2043, in embedding
    return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
RuntimeError: CUDA error: device-side assert triggered
qiudao767 commented 3 years ago

vocab.json:

{"<PAD>": 0, "<SOS>": 1, "<EOS>": 2, "<UNK>": 3, "2": 4, "\\frac": 5, "{": 6, "9": 7, "}": 8, "1": 9, "0": 10, "8": 11, "p": 12, "\\pm": 13, "7": 14, "<": 15, "d": 16, "\\sqrt": 17, "\\xi": 18, "\\varphi": 19, "3": 20, "5": 21, "[": 22, "]": 23, "Q": 24, "\\sec": 25, "^": 26, "m": 27, "\\subseteq": 28, "U": 29, "6": 30, "\\mp": 31, "\\beta": 32, "\\supseteq": 33, "b": 34, "4": 35, "l": 36, "\\div": 37, "\\csc": 38, "\\circ": 39, ">": 40, "n": 41, "x": 42, "N": 43, "a": 44, "\\tan": 45, "D": 46, "\\ge": 47, "h": 48, "T": 49, "H": 50, "-": 51, "r": 52, "+": 53, "\\supset": 54, "I": 55, "\\sin": 56, "\\pi": 57, "J": 58, "\\cot": 59, "=": 60, "W": 61, "v": 62, "o": 63, "\\ni": 64, "\\alpha": 65, "\\in": 66, "y": 67, "L": 68, "\\cos": 69, "q": 70, "\\theta": 71, "z": 72, "V": 73, "K": 74, "P": 75, "X": 76, "\\le": 77, "M": 78, "O": 79, "w": 80, "F": 81, "R": 82, "\\times": 83, "C": 84, "Z": 85, "s": 86, "S": 87, "E": 88, "Y": 89, "u": 90, "A": 91, "\\subset": 92, "G": 93, "g": 94, "i": 95, "j": 96, "B": 97, "c": 98, "k": 99, "t": 100, "f": 101}