search

ZJULearning / pixel_link

Implementation of our paper 'PixelLink: Detecting Scene Text via Instance Segmentation' in AAAI2018
MIT License
767 stars 254 forks source link

InternalError (see above for traceback): Blas SGEMM launch failed : m=61440, n=2, k=256 #163

Open DucLong06 opened 4 years ago

DucLong06 commented 4 years ago

I want to run test_pixel_link_on_any_image.py i use win 10os anaconda3 cuda9 tensorflow 1.5.0

python test_pixel_link_on_any_image.py --checkpoint_path C:\Users\Administrator\Desktop\LOG\pixel_link\pixel_link_vgg_4s\conv3_3 --dataset_dir C:\Users\Administrator\Desktop\LOG\pixel_link\data --eval_image_width=1280 --eval_image_height=768 --pixel_conf_threshold=0.5 --link_conf_threshold=0.5 --gpu_memory_fraction=-1

i have error :

WARNING:tensorflow:From test_pixel_link_on_any_image.py:76: get_or_create_global_step (from tensorflow.contrib.framework.python.ops.variables) is deprecated and will be removed in a future version. Instructions for updating: Please switch to tf.train.get_or_create_global_step 2020-04-01 16:32:53.590041: I C:\tf_jenkins\workspace\rel-win\M\windows-gpu\PY\35\tensorflow\core\platform\cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX AVX2 2020-04-01 16:32:54.080675: I C:\tf_jenkins\workspace\rel-win\M\windows-gpu\PY\35\tensorflow\core\common_runtime\gpu\gpu_device.cc:1105] Found device 0 with properties: name: GeForce GTX 1060 6GB major: 6 minor: 1 memoryClockRate(GHz): 1.7335 pciBusID: 0000:01:00.0 totalMemory: 6.00GiB freeMemory: 4.96GiB 2020-04-01 16:32:54.085798: I C:\tf_jenkins\workspace\rel-win\M\windows-gpu\PY\35\tensorflow\core\common_runtime\gpu\gpu_device.cc:1195] Creating TensorFlow device (/device:GPU:0) -> (device: 0, name: GeForce GTX 1060 6GB, pci bus id: 0000:01:00.0, compute capability: 6.1) INFO:tensorflow:Restoring parameters from C:/Users/Administrator/Desktop/LOG/pixel_link/pixel_link_vgg_4s/conv3_3/model.ckpt-38055 2020-04-01 16:34:52.911629: E C:\tf_jenkins\workspace\rel-win\M\windows-gpu\PY\35\tensorflow\stream_executor\cuda\cuda_blas.cc:444] failed to create cublas handle: CUBLAS_STATUS_NOT_INITIALIZED 2020-04-01 16:34:52.943963: W C:\tf_jenkins\workspace\rel-win\M\windows-gpu\PY\35\tensorflow\stream_executor\stream.cc:1901] attempting to perform BLAS operation using StreamExecutor without BLAS support Traceback (most recent call last): File "C:\Anaconda3\lib\site-packages\tensorflow\python\client\session.py", line 1350, in _do_call return fn(*args) File "C:\Anaconda3\lib\site-packages\tensorflow\python\client\session.py", line 1329, in _run_fn status, run_metadata) File "C:\Anaconda3\lib\site-packages\tensorflow\python\framework\errors_impl.py", line 473, in exit c_api.TF_GetCode(self.status.status)) tensorflow.python.framework.errors_impl.InternalError: Blas SGEMM launch failed : m=61440, n=2, k=256 [[Node: evaluation_768x1280/pixel_cls/score_from_conv3_3/Conv2D = Conv2D[T=DT_FLOAT, data_format="NHWC", dilations=[1, 1, 1, 1], padding="SAME", strides=[1, 1, 1, 1], use_cudnn_on_gpu=true, _device="/job:localhost/replica:0/task:0/device:GPU:0"](evaluation_768x1280/conv3/conv3_3/Relu, pixel_cls/score_from_conv3_3/weights/read/_193)]] [[Node: evaluation_768x1280/Reshape/_195 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_209_evaluation_768x1280/Reshape", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"]()]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "test_pixel_link_on_any_image.py", line 157, in tf.app.run() File "C:\Anaconda3\lib\site-packages\tensorflow\python\platform\app.py", line 124, in run _sys.exit(main(argv)) File "test_pixel_link_on_any_image.py", line 153, in main test() File "test_pixel_link_on_any_image.py", line 120, in test feed_dict={image: image_data}) File "C:\Anaconda3\lib\site-packages\tensorflow\python\client\session.py", line 895, in run run_metadata_ptr) File "C:\Anaconda3\lib\site-packages\tensorflow\python\client\session.py", line 1128, in _run feed_dict_tensor, options, run_metadata) File "C:\Anaconda3\lib\site-packages\tensorflow\python\client\session.py", line 1344, in _do_run options, run_metadata) File "C:\Anaconda3\lib\site-packages\tensorflow\python\client\session.py", line 1363, in _do_call raise type(e)(node_def, op, message) tensorflow.python.framework.errors_impl.InternalError: Blas SGEMM launch failed : m=61440, n=2, k=256 [[Node: evaluation_768x1280/pixel_cls/score_from_conv3_3/Conv2D = Conv2D[T=DT_FLOAT, data_format="NHWC", dilations=[1, 1, 1, 1], padding="SAME", strides=[1, 1, 1, 1], use_cudnn_on_gpu=true, _device="/job:localhost/replica:0/task:0/device:GPU:0"](evaluation_768x1280/conv3/conv3_3/Relu, pixel_cls/score_from_conv3_3/weights/read/_193)]] [[Node: evaluation_768x1280/Reshape/_195 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_209_evaluation_768x1280/Reshape", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"]()]]

Caused by op 'evaluation_768x1280/pixel_cls/score_from_conv3_3/Conv2D', defined at: File "test_pixel_link_on_any_image.py", line 157, in tf.app.run() File "C:\Anaconda3\lib\site-packages\tensorflow\python\platform\app.py", line 124, in run _sys.exit(main(argv)) File "test_pixel_link_on_any_image.py", line 153, in main test() File "test_pixel_link_on_any_image.py", line 88, in test net = pixel_link_symbol.PixelLinkNet(b_image, is_training=False) File "C:\Users\Administrator\Desktop\LOG\pixel_link\nets\pixel_link_symbol.py", line 18, in init self._fuse_feat_layers() File "C:\Users\Administrator\Desktop\LOG\pixel_link\nets\pixel_link_symbol.py", line 150, in _fuse_feat_layers config.num_classes, scope = 'pixel_cls') File "C:\Users\Administrator\Desktop\LOG\pixel_link\nets\pixel_link_symbol.py", line 137, in _fuse_by_cascade_conv1x1_upsample_sum num_classes, current_layer_name) File "C:\Users\Administrator\Desktop\LOG\pixel_link\nets\pixel_link_symbol.py", line 58, in _score_layer normalizer_fn=None) File "C:\Anaconda3\lib\site-packages\tensorflow\contrib\framework\python\ops\arg_scope.py", line 182, in func_with_args return func(*args, current_args) File "C:\Anaconda3\lib\site-packages\tensorflow\contrib\layers\python\layers\layers.py", line 1057, in convolution outputs = layer.apply(inputs) File "C:\Anaconda3\lib\site-packages\tensorflow\python\layers\base.py", line 762, in apply return self.call(inputs, *args, *kwargs) File "C:\Anaconda3\lib\site-packages\tensorflow\python\layers\base.py", line 652, in call outputs = self.call(inputs, args, kwargs) File "C:\Anaconda3\lib\site-packages\tensorflow\python\layers\convolutional.py", line 167, in call outputs = self._convolution_op(inputs, self.kernel) File "C:\Anaconda3\lib\site-packages\tensorflow\python\ops\nn_ops.py", line 838, in call return self.conv_op(inp, filter) File "C:\Anaconda3\lib\site-packages\tensorflow\python\ops\nn_ops.py", line 502, in call return self.call(inp, filter) File "C:\Anaconda3\lib\site-packages\tensorflow\python\ops\nn_ops.py", line 190, in call name=self.name) File "C:\Anaconda3\lib\site-packages\tensorflow\python\ops\gen_nn_ops.py", line 725, in conv2d data_format=data_format, dilations=dilations, name=name) File "C:\Anaconda3\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 787, in _apply_op_helper op_def=op_def) File "C:\Anaconda3\lib\site-packages\tensorflow\python\framework\ops.py", line 3160, in create_op op_def=op_def) File "C:\Anaconda3\lib\site-packages\tensorflow\python\framework\ops.py", line 1625, in init self._traceback = self._graph._extract_stack() # pylint: disable=protected-access

InternalError (see above for traceback): Blas SGEMM launch failed : m=61440, n=2, k=256 [[Node: evaluation_768x1280/pixel_cls/score_from_conv3_3/Conv2D = Conv2D[T=DT_FLOAT, data_format="NHWC", dilations=[1, 1, 1, 1], padding="SAME", strides=[1, 1, 1, 1], use_cudnn_on_gpu=true, _device="/job:localhost/replica:0/task:0/device:GPU:0"](evaluation_768x1280/conv3/conv3_3/Relu, pixel_cls/score_from_conv3_3/weights/read/_193)]] [[Node: evaluation_768x1280/Reshape/_195 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_209_evaluation_768x1280/Reshape", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"]()]]