Error: while running main.py cudnn error

[ArditHalimi]

Hello, I have getting this error while running main.py in gpu CUDNN_STATUS_INTERNAL_ERROR

I have added this config in Pose/pose_visualizer.py but same error

        config_proto = tf.ConfigProto()
        config_proto.gpu_options.allow_growth = True
        self.persistent_sess = tf.Session(graph=self.graph, config=config_proto)

2020-04-15 15:41:41.483304: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudnn.so.7 2020-04-15 15:41:41.955955: E tensorflow/stream_executor/cuda/cuda_dnn.cc:329] Could not create cudnn handle: CUDNN_STATUS_INTERNAL_ERROR 2020-04-15 15:41:41.961271: E tensorflow/stream_executor/cuda/cuda_dnn.cc:329] Could not create cudnn handle: CUDNN_STATUS_INTERNAL_ERROR Traceback (most recent call last): File "/home/ardit/anaconda3/envs/pose_est/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1356, in _do_call return fn(*args) File "/home/ardit/anaconda3/envs/pose_est/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1341, in _run_fn options, feed_dict, fetch_list, target_list, run_metadata) File "/home/ardit/anaconda3/envs/pose_est/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1429, in _call_tf_sessionrun run_metadata) tensorflow.python.framework.errors_impl.UnknownError: 2 root error(s) found. (0) Unknown: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above. [[{{node TfPoseEstimator/conv1_1/Conv2D}}]] [[TfPoseEstimator/Openpose/concat_stage7/_3]] (1) Unknown: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above. [[{{node TfPoseEstimator/conv1_1/Conv2D}}]] 0 successful operations. 0 derived errors ignored.

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "main.py", line 41, in humans = estimator.inference(show) File "/home/ardit/Documents/human_pose/Online-Realtime-Action-Recognition-based-on-OpenPose/Pose/pose_visualizer.py", line 107, in inference output = self.persistent_sess.run(self.tensor_output, feed_dict={self.tensor_image: rois}) File "/home/ardit/anaconda3/envs/pose_est/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 950, in run run_metadata_ptr) File "/home/ardit/anaconda3/envs/pose_est/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1173, in _run feed_dict_tensor, options, run_metadata) File "/home/ardit/anaconda3/envs/pose_est/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1350, in _do_run run_metadata) File "/home/ardit/anaconda3/envs/pose_est/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1370, in _do_call raise type(e)(node_def, op, message) tensorflow.python.framework.errors_impl.UnknownError: 2 root error(s) found. (0) Unknown: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above. [[node TfPoseEstimator/conv1_1/Conv2D (defined at /home/ardit/Documents/human_pose/Online-Realtime-Action-Recognition-based-on-OpenPose/Pose/pose_visualizer.py:23) ]] [[TfPoseEstimator/Openpose/concat_stage7/_3]] (1) Unknown: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above. [[node TfPoseEstimator/conv1_1/Conv2D (defined at /home/ardit/Documents/human_pose/Online-Realtime-Action-Recognition-based-on-OpenPose/Pose/pose_visualizer.py:23) ]] 0 successful operations. 0 derived errors ignored.

Original stack trace for 'TfPoseEstimator/conv1_1/Conv2D': File "main.py", line 15, in estimator = load_pretrain_model('VGG_origin') File "/home/ardit/Documents/human_pose/Online-Realtime-Action-Recognition-based-on-OpenPose/utils.py", line 48, in load_pretrain_model return TfPoseVisualizer(graph_path, target_size=(input_width, input_height)) File "/home/ardit/Documents/human_pose/Online-Realtime-Action-Recognition-based-on-OpenPose/Pose/pose_visualizer.py", line 23, in init tf.import_graph_def(graph_def, name='TfPoseEstimator') File "/home/ardit/anaconda3/envs/pose_est/lib/python3.7/site-packages/tensorflow/python/util/deprecation.py", line 507, in new_func return func(*args, **kwargs) File "/home/ardit/anaconda3/envs/pose_est/lib/python3.7/site-packages/tensorflow/python/framework/importer.py", line 443, in import_graph_def _ProcessNewOps(graph) File "/home/ardit/anaconda3/envs/pose_est/lib/python3.7/site-packages/tensorflow/python/framework/importer.py", line 236, in _ProcessNewOps for new_op in graph._add_new_tf_operations(compute_devices=False): # pylint: disable=protected-access File "/home/ardit/anaconda3/envs/pose_est/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 3751, in _add_new_tf_operations for c_op in c_api_util.new_tf_operations(self) File "/home/ardit/anaconda3/envs/pose_est/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 3751, in for c_op in c_api_util.new_tf_operations(self) File "/home/ardit/anaconda3/envs/pose_est/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 3641, in _create_op_from_tf_operation ret = Operation(c_op, self) File "/home/ardit/anaconda3/envs/pose_est/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 2005, in init self._traceback = tf_stack.extract_stack()

[ArditHalimi]

fixed by adding config to Tracking/generate_dets.py line 75

        config_proto = tf.ConfigProto()
        config_proto.gpu_options.allow_growth = True

[whwtraffic12306]

@ArditHalimi ，Excuse me, how many lines do you add these three lines of code to?

    config_proto = tf.ConfigProto()
    config_proto.gpu_options.allow_growth = True
    self.persistent_sess = tf.Session(graph=self.graph, config=config_proto)