TobyPDE / FRRN

Full Resolution Residual Networks for Semantic Image Segmentation
MIT License
278 stars 93 forks source link

Getting from cudnn #35

Closed rnunziata closed 6 years ago

rnunziata commented 6 years ago

Is this a version error incompatibility error? This is using the mypredict code for single image processing.

Using cuDNN version 5110 on context None
Mapped name None to device cuda0: GeForce GTX 1060 (0000:01:00.0)
2017-09-23 17:11:07,721 INFO Only predict function
Images 1
Traceback (most recent call last):
  File "predict.py", line 167, in <module>
    main()
  File "predict.py", line 142, in main
    predictions = pred_fn(image)
  File "/usr/local/lib/python2.7/dist-packages/theano/compile/function_module.py", line 917, in __call__
    storage_map=getattr(self.fn, 'storage_map', None))
  File "/usr/local/lib/python2.7/dist-packages/theano/gof/link.py", line 325, in raise_with_op
    reraise(exc_type, exc_value, exc_trace)
  File "/usr/local/lib/python2.7/dist-packages/theano/compile/function_module.py", line 903, in __call__
    self.fn() if output_subset is None else\
RuntimeError: error getting worksize: CUDNN_STATUS_BAD_PARAM
Apply node that caused the error: GpuDnnConv{algo='small', inplace=True, num_groups=1}(GpuContiguous.0, GpuContiguous.0, GpuAllocEmpty{dtype='float32', context_name=None}.0, GpuDnnConvDesc{border_mode='half', subsample=(1, 1), dilation=(1, 1), conv_mode='conv', precision='float64', num_groups=1}.0, Constant{1.0}, Constant{0.0})
Toposort index: 757
Inputs types: [GpuArrayType<None>(float32, 4D), GpuArrayType<None>(float64, 4D), GpuArrayType<None>(float32, 4D), <theano.gof.type.CDataType object at 0x7f173b6f5b50>, Scalar(float32), Scalar(float32)]
Inputs shapes: [(1, 3, 256, 512), (48, 3, 5, 5), (1, 48, 256, 512), 'No shapes', (), ()]
Inputs strides: [(1572864, 524288, 2048, 4), (600, 200, 40, 8), (25165824, 524288, 2048, 4), 'No strides', (), ()]
Inputs values: ['not shown', 'not shown', 'not shown', <capsule object NULL at 0x7f1731d45a80>, 1.0, 0.0]
Outputs clients: [[GpuElemwise{Composite{(i0 * (Composite{(((i0 - i1) * i2 * i3) + i4)}(i1, i2, i3, i4, i5) + Abs(Composite{(((i0 - i1) * i2 * i3) + i4)}(i1, i2, i3, i4, i5))))}}[]<gpuarray>(GpuArrayConstant{[[[[ 0.5]]]]}, GpuDnnConv{algo='small', inplace=True, num_groups=1}.0, InplaceGpuDimShuffle{x,0,x,x}.0, InplaceGpuDimShuffle{x,0,x,x}.0, InplaceGpuDimShuffle{x,0,x,x}.0, InplaceGpuDimShuffle{x,0,x,x}.0)]]

Backtrace when the node is created(use Theano flag traceback.limit=N to make it longer):
  File "predict.py", line 167, in <module>
    main()
  File "predict.py", line 81, in main
    pred_fn = train.only_predict_function(network)
  File "/home/rjn/opencv3-p3-code/classification_and_boxing/newstuff/onhold/FRRN/FRRN/train.py", line 181, in only_predict_function
    network.output_layers, deterministic=True)[0]
  File "/usr/local/lib/python2.7/dist-packages/lasagne/layers/helper.py", line 197, in get_output
    all_outputs[layer] = layer.get_output_for(layer_inputs, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/lasagne/layers/conv.py", line 333, in get_output_for
    conved = self.convolve(input, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/lasagne/layers/conv.py", line 611, in convolve
    filter_flip=self.flip_filters)

HINT: Use the Theano flag 'exception_verbosity=high' for a debugprint and storage map footprint of this apply node.
rnunziata commented 6 years ago

resolved....reinstalled theano from source