Runtime error pytorch 0.3.1,python3

[lxtGH]

/opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [704,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [192,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [576,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [960,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [832,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [448,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [421,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [422,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [942,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [320,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [349,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [313,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [314,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [815,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [821,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [822,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [823,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [699,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [700,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [571,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [64,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [65,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [66,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [67,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [68,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [69,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [70,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [71,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [72,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [73,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [74,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [75,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [76,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [77,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [78,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [79,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [80,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [81,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [82,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [83,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [84,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [85,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [86,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [87,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [88,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [89,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [90,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [91,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [92,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [93,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [94,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [95,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [32,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [33,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [34,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [35,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [36,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [37,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [38,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [39,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [40,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [41,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [42,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [43,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [44,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [45,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [46,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [47,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [48,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [49,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [50,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [51,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [52,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [53,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [54,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [55,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [56,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [57,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [58,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [59,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [60,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [61,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [62,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [63,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [160,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [161,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [162,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [163,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [164,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [165,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [166,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [167,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [168,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [169,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [170,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [171,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [172,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [173,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [174,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [175,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [176,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [177,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [178,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [179,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [180,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [181,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [182,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [183,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [184,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [185,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [186,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [187,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [188,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [189,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [190,0,0] Assertion t >= 0 && t < n_classes failed. /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T , T , T , long , T , int, int, int, int, int, long) [with T = float, AccumT = float]: block: [9,0,0], thread: [191,0,0] Assertion t >= 0 && t < n_classes failed. Traceback (most recent call last): File "main.py", line 409, in trainValidateSegmentation(parser.parse_args()) File "main.py", line 335, in trainValidateSegmentation train(args, trainLoader_scale1, model, criteria, optimizer, epoch) File "main.py", line 105, in train loss.backward() File "/home/lxt/anaconda3/lib/python3.6/site-packages/torch/autograd/variable.py", line 167, in backward torch.autograd.backward(self, gradient, retain_graph, create_graph, retain_variables) File "/home/lxt/anaconda3/lib/python3.6/site-packages/torch/autograd/init.py", line 99, in backward variables, grad_variables, retain_graph) RuntimeError: CUDNN_STATUS_INTERNAL_ERROR

When I train the model use python main.py --scaleIn 8

[sacmehta]

The issue seems to be with the PyTorch/CUDA/CUDNN installation . Please post it in relevant forums.

[lxtGH]

Hi ! nice work @sacmehta , but you should note that your code have label map 255 into 19, it takes me an hour to fix this error, it is not CUDA/CUDNN installation error. The Assertion note that the range of class labels. I think we should add relabel function like this: label[label == 255 ] = 19

[sacmehta]

Thanks. I somehow missed the assertion error.

Well, hard coding the undefined classes such as 255 to a particular class is not good idea. For example, class 19 represents background in the Cityscapes dataset while class 0 represents background in the PASCAL dataset.

If you run the Cityscapes processing scripts to generate the training label images, then you would not encounter such issue.

Anyways, I will list it as a FAQ.

[lxtGH]

aha, I used the cityscapes scripts, after that the label is still 255(white color), and aslo I think a potential error is memory limit because your code only use one GPU, so what about add this line: model = torch.nn.DataParallel(model, device_ids=range(torch.cuda.device_count())) @sacmehta

[sacmehta]

We suggested to use DataParallel wrapper for multiple GPUs. See README.MD inside train directory.

Please feel free to customize the code as per your needs.

[bjchen666]

@lxtGH 你好，我也碰到你这个问题，CUDNN_STATUS_INTERNAL_ERROR，不知道你解决了没有，我解决不了这个问题。

[lxtGH]

@gshdong please check your image label you load. 检查下你的输入图像标注是否合理

[acgtyrant]

@sacmehta I am amazing that you train the model with the 255 label in cityscapes while the other model ignore it immediately.

[sacmehta]

@acgtyrant Sorry, I am not able to understand your question. Could you be a bit more specific?

[acgtyrant]

You did not ignore this index in nn.NLLLoss2d.

ignore_index (int, optional) – Specifies a target value that is ignored and does not contribute to the input gradient.

The cityscapes evaluation server does not evaluate this index too, so the others researcher ignore this index in their criterion.

[sacmehta]

Yes, we do not ignore that index. Ignoring that index might bump up the accuracy by a point, but that leads to poor generalization in the wild. You can see our paper for more details.