Report mistakes during training,please please please help

[Anonymoussubmitter123]

Training: 0%| | 6/900000 [00:02<122:44:49, 2.04it/s]Traceback (most recent call last): File "train.py", line 234, in main(args, configs) File "train.py", line 108, in main output = model((exe_batch[2:])) File "/root/anaconda3/envs/Adaspeech/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl result = self.forward(input, kwargs) File "/root/anaconda3/envs/Adaspeech/lib/python3.7/site-packages/torch/nn/parallel/data_parallel.py", line 159, in forward return self.module(*inputs[0], *kwargs[0]) File "/root/anaconda3/envs/Adaspeech/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl result = self.forward(input, kwargs) File "/AdaSpeech-main/model/adaspeech.py", line 75, in forward output = self.encoder(texts, speaker_embedding, src_masks) File "/root/anaconda3/envs/Adaspeech/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl result = self.forward(*input, kwargs) File "/AdaSpeech-main/transformer/Models.py", line 95, in forward enc_output, speaker_embedding, mask=mask, slf_attn_mask=slf_attn_mask File "/root/anaconda3/envs/Adaspeech/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl result = self.forward(*input, *kwargs) File "/AdaSpeech-main/transformer/Layers.py", line 27, in forward enc_output = self.pos_ffn(enc_output, speaker_embedding) File "/root/anaconda3/envs/Adaspeech/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl result = self.forward(input, kwargs) File "/AdaSpeech-main/transformer/SubLayers.py", line 106, in forward output = self.w_2(F.relu(self.w_1(output))) File "/root/anaconda3/envs/Adaspeech/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl result = self.forward(*input, **kwargs) File "/root/anaconda3/envs/Adaspeech/lib/python3.7/site-packages/torch/nn/modules/conv.py", line 259, in forward self.padding, self.dilation, self.groups) RuntimeError: CUDA error: device-side assert triggered /pytorch/aten/src/ATen/native/cuda/Indexing.cu:605: indexSelectSmallIndex: block: [1,0,0], thread: [0,0,0] Assertion srcIndex < srcSelectDimSize failed. /pytorch/aten/src/ATen/native/cuda/Indexing.cu:605: indexSelectSmallIndex: block: [1,0,0], thread: [1,0,0] Assertion srcIndex < srcSelectDimSize failed. /pytorch/aten/src/ATen/native/cuda/Indexing.cu:605: indexSelectSmallIndex: block: [1,0,0], thread: [2,0,0] Assertion srcIndex < srcSelectDimSize failed. /pytorch/aten/src/ATen/native/cuda/Indexing.cu:605: indexSelectSmallIndex: block: [1,0,0], thread: [3,0,0] Assertion srcIndex < srcSelectDimSize failed.