eriklindernoren / PyTorch-YOLOv3

Minimal PyTorch implementation of YOLOv3
GNU General Public License v3.0
7.32k stars 2.63k forks source link

RuntimeError: CUDA error: an illegal memory access was encountered #244

Open sholalkere opened 5 years ago

sholalkere commented 5 years ago

Happens here: class_mask[b, best_n, gj, gi] = (pred_cls[b, best_n, gj, gi].argmax(-1) == target_labels).float() in utils file.

My env: Python 3.7.3 Pytorch 1.0.0 Cudatoolkit 10.1.168

Windows 10 i5 cpu and 4gb 1050(xps 9560) 32 gb ram

changed batch_size to 1, num_cpu to 4, img_size to 200

Full error:

Namespace(batch_size=1, checkpoint_interval=1, compute_map=False, data_config='config/custom.data', epochs=100, evaluation_interval=1, gradient_accumulations=2, img_size=200, model_def='config/yolov3-custom.cfg', multiscale_training=True, n_cpu=4, pretrained_weights=None)
Traceback (most recent call last):
  File "train.py", line 105, in <module>
    loss, outputs = model(imgs, targets)
  File "C:\Users\sidh\.conda\envs\kuzu\lib\site-packages\torch\nn\modules\module.py", line 489, in __call__
    result = self.forward(*input, **kwargs)
  File "C:\Users\sidh\kaggle\kuzushiji\working\PyTorch-YOLOv3-master\models.py", line 259, in forward
    x, layer_loss = module[0](x, targets, img_dim)
  File "C:\Users\sidh\.conda\envs\kuzu\lib\site-packages\torch\nn\modules\module.py", line 489, in __call__
    result = self.forward(*input, **kwargs)
  File "C:\Users\sidh\kaggle\kuzushiji\working\PyTorch-YOLOv3-master\models.py", line 188, in forward
    ignore_thres=self.ignore_thres,
  File "C:\Users\sidh\kaggle\kuzushiji\working\PyTorch-YOLOv3-master\utils\utils.py", line 318, in build_targets
    class_mask[b, best_n, gj, gi] = (pred_cls[b, best_n, gj, gi].argmax(-1) == target_labels).float()
RuntimeError: CUDA error: an illegal memory access was encountered
Flova commented 3 years ago

Is this issue still relevant/occurring?