Closed jpsml closed 5 years ago
I have solved it by changing line 76 of "maskfusion/Core/CMakeLists.txt" according to the CUDA archictecture of the GPU I am using (GeForce RTX 2080 Ti), which is 7.5.
So, I replaced this line:
set(CUDA_ARCH_BIN "50 52 61" CACHE STRING "Specify 'real' GPU arch to build binaries for, BIN(PTX) format is supported. Example: 1.3 2.1(1.3) or 13 21(13)")
By this line:
set(CUDA_ARCH_BIN "75" CACHE STRING "Specify 'real' GPU arch to build binaries for, BIN(PTX) format is supported. Example: 1.3 2.1(1.3) or 13 21(13)")
Hello, may I ask how to solve this problem in 3060 graphics card power 8.6 situation
I am trying to run MaskFusion with the teddy-handover.klg sequence from Co-Fusion available at http://visual.cs.ucl.ac.uk/pubs/cofusion/data/teddy-handover.klg. In order to run it, I needed to change line 52 of maskfusion/build/GUI/MaskRCNN.py as following:
from keras.backend.tensorflow_backend import set_session, clear_session
When I run it with the command "./MaskFusion -l ../../teddy-handover.klg", a window appears, but when I click the "Pause" option to start tracking, the following error occurs:
/home/jpsml/maskfusion/Core/Cuda/cudafuncs.cu(646) : cudaSafeCall() Runtime API error : invalid texture reference. /home/jpsml/maskfusion/Core/GPUTexture.cpp(73) : cudaSafeCall() Runtime API error : invalid texture reference.
I am using Ubuntu 19.04, CUDA 10.0, TensorFlow 1.14, Keras 2.3.
You can find below the complete log:
jpsml@jpsml-desktop:~/maskfusion/build/GUI$ ./MaskFusion -l ../../teddy-handover.klg Calibration set to resolution: 640x480, [fx: 528 fy: 528, cx: 320 cy: 240] Reading log file: ../../teddy-handover.klg which has 528 frames. Initialised MainController. Frame resolution is set to: 640x480 Exporting results to: ../../teddy-handover.klg-export//
WARNING:tensorflow:From /home/jpsml/maskfusion/build/GUI/MaskRCNN.py:51: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.
2019-09-20 11:11:39.509898: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2019-09-20 11:11:39.533194: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 3696000000 Hz 2019-09-20 11:11:39.534004: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f13b9042da0 executing computations on platform Host. Devices: 2019-09-20 11:11:39.534020: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (0):,
2019-09-20 11:11:39.534089: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcuda.so.1
2019-09-20 11:11:39.534157: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2019-09-20 11:11:39.534591: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1640] Found device 0 with properties:
name: GeForce RTX 2080 Ti major: 7 minor: 5 memoryClockRate(GHz): 1.635
pciBusID: 0000:01:00.0
2019-09-20 11:11:39.534777: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudart.so.10.0
2019-09-20 11:11:39.535622: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcublas.so.10.0
2019-09-20 11:11:39.536360: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcufft.so.10.0
2019-09-20 11:11:39.536554: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcurand.so.10.0
2019-09-20 11:11:39.537570: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcusolver.so.10.0
2019-09-20 11:11:39.538342: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcusparse.so.10.0
2019-09-20 11:11:39.538395: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Could not dlopen library 'libcudnn.so.7'; dlerror: libcudnn.so.7: cannot open shared object file: No such file or directory
2019-09-20 11:11:39.538401: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1663] Cannot dlopen some GPU libraries. Skipping registering GPU devices...
2019-09-20 11:11:39.538463: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1181] Device interconnect StreamExecutor with strength 1 edge matrix:
2019-09-20 11:11:39.538471: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1187] 0
2019-09-20 11:11:39.538474: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1200] 0: N
2019-09-20 11:11:39.539481: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2019-09-20 11:11:39.539973: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f13ba99a3c0 executing computations on platform CUDA. Devices:
2019-09-20 11:11:39.539985: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (0): GeForce RTX 2080 Ti, Compute Capability 7.5
Using TensorFlow backend.
Configurations: BACKBONE resnet101 BACKBONE_STRIDES [4, 8, 16, 32, 64] BATCH_SIZE 1 BBOX_STD_DEV [0.1 0.1 0.2 0.2] COMPUTE_BACKBONE_SHAPE None DETECTION_MAX_INSTANCES 100 DETECTION_MIN_CONFIDENCE 0.7 DETECTION_NMS_THRESHOLD 0.3 FPN_CLASSIF_FC_LAYERS_SIZE 1024 GPU_COUNT 1 GRADIENT_CLIP_NORM 5.0 IMAGES_PER_GPU 1 IMAGE_CHANNEL_COUNT 3 IMAGE_MAX_DIM 1024 IMAGE_META_SIZE 93 IMAGE_MIN_DIM 800 IMAGE_MIN_SCALE 0 IMAGE_RESIZE_MODE square IMAGE_SHAPE [1024 1024 3] LEARNING_MOMENTUM 0.9 LEARNING_RATE 0.001 LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0} MASK_POOL_SIZE 14 MASK_SHAPE [28, 28] MAX_GT_INSTANCES 100 MEAN_PIXEL [123.7 116.8 103.9] MINI_MASK_SHAPE (56, 56) NAME coco NUM_CLASSES 81 POOL_SIZE 7 POST_NMS_ROIS_INFERENCE 1000 POST_NMS_ROIS_TRAINING 2000 PRE_NMS_LIMIT 6000 ROI_POSITIVE_RATIO 0.33 RPN_ANCHOR_RATIOS [0.5, 1, 2] RPN_ANCHOR_SCALES (32, 64, 128, 256, 512) RPN_ANCHOR_STRIDE 1 RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2] RPN_NMS_THRESHOLD 0.7 RPN_TRAIN_ANCHORS_PER_IMAGE 256 STEPS_PER_EPOCH 1000 TOP_DOWN_PYRAMID_SIZE 256 TRAIN_BN False TRAIN_ROIS_PER_IMAGE 200 USE_MINI_MASK True USE_RPN_ROIS True VALIDATION_STEPS 50 WEIGHT_DECAY 0.0001
WARNING:tensorflow:From /home/jpsml/maskfusion/python-environment/lib/python3.7/site-packages/keras/backend/tensorflow_backend.py:4070: The name tf.nn.max_pool is deprecated. Please use tf.nn.max_pool2d instead.
WARNING:tensorflow:From /home/jpsml/maskfusion/deps/Mask_RCNN/mrcnn/model.py:341: The name tf.log is deprecated. Please use tf.math.log instead.
WARNING:tensorflow:From /home/jpsml/maskfusion/deps/Mask_RCNN/mrcnn/model.py:399: add_dispatch_support..wrapper (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where
WARNING:tensorflow:From /home/jpsml/maskfusion/deps/Mask_RCNN/mrcnn/model.py:423: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version.
Instructions for updating:
box_ind is deprecated, use box_indices instead
WARNING:tensorflow:From /home/jpsml/maskfusion/deps/Mask_RCNN/mrcnn/model.py:720: The name tf.sets.set_intersection is deprecated. Please use tf.sets.intersection instead.
WARNING:tensorflow:From /home/jpsml/maskfusion/deps/Mask_RCNN/mrcnn/model.py:722: The name tf.sparse_tensor_to_dense is deprecated. Please use tf.sparse.to_dense instead.
WARNING:tensorflow:From /home/jpsml/maskfusion/deps/Mask_RCNN/mrcnn/model.py:772: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use
tf.cast
instead. 2019-09-20 11:11:44.478694: W tensorflow/compiler/jit/mark_for_compilation_pass.cc:1412] (One-time warning): Not using XLA:CPU for cluster because envvar TF_XLA_FLAGS=--tf_xla_cpu_global_jit was not set. If you want XLA:CPU, either set that envvar, or use experimental_jit_scope to enable XLA:CPU. To confirm that XLA is active, pass --vmodule=xla_compilation_cache=1 (as a proper command-line flag, not via TF_XLA_FLAGS) or set the envvar XLA_FLAGS=--xla_hlo_profile./home/jpsml/maskfusion/Core/Cuda/cudafuncs.cu(646) : cudaSafeCall() Runtime API error : invalid texture reference. /home/jpsml/maskfusion/Core/GPUTexture.cpp(73) : cudaSafeCall() Runtime API error : invalid texture reference.
Do you know how should I proceed in order to run it? Thank you.