RandLA-Net training not possible on RTX 3070?

[IMSHJI]

Hello. Thank you for your RandLA-Net contribution. I tried RandLA-Net using only the cpu, but the learning rate is very slow, so I want to utilize the gpu.

Currently, it is possible to learn without problems on the GTX 1660 Ti.

(In order to speed up the learning and review the performance of RandLA-Net, only hallway data from Areas 1 to 6 were used, but result models were not created.

There was an error saying that there was no data in results/Log-%%/snapshots/ , but looking at the train_log, the training was successful, and the related folder was also created, but only the model data (.meta etc) did not seem to be created.

_/home/shji/anaconda3/envs/ranet/lib/python3.6/site-packages/tensorflow/python/ops/gradients_impl.py:108: UserWarning: Converting sparse IndexedSlices to a dense Tensor of unknown shape. This may consume a large amount of memory. "Converting sparse IndexedSlices to a dense Tensor of unknown shape. " Traceback (most recent call last): File "main_S3DIS.py", line 250, in snap_steps = [int(f[:-5].split('-')[-1]) for f in os.listdir(snap_path) if f[-5:] == '.meta'] FileNotFoundError: [Errno 2] No such file or directory: 'results/Log_2021-08-09_05-05-33/snapshots'_

However, for Area2, the training model and test data were generated. I'm curious as to why this error occurred.)

I'm trying to learn on an RTX 3070. I am using Python 3.6 CUDA 9.0 cuDNN 7.4.1 Tensorflow-gpu 1.11 and when I try sh jobs_6_fold_cv_s3dis.sh I get the following error.

_(ranet) shji@shji-rtx3070:~/Documents/RandLA-Net$ time sh jobs_6_fold_cv_s3dis.sh Area_5_office_36_KDTree.pkl 2.3 MB loaded in 0.0s Area_1_office_12_KDTree.pkl 1.6 MB loaded in 0.0s Area_4_hallway_4_KDTree.pkl 2.2 MB loaded in 0.0s Area_4_hallway_13_KDTree.pkl 1.7 MB loaded in 0.0s Area_6_office_21_KDTree.pkl 1.1 MB loaded in 0.0s Area_5_hallway_6_KDTree.pkl 2.0 MB loaded in 0.0s Area_4_office_14_KDTree.pkl 0.5 MB loaded in 0.0s Area_5_WC_1_KDTree.pkl 1.3 MB loaded in 0.0s Area_5_office_19_KDTree.pkl 3.1 MB loaded in 0.0s Area_6_office_9_KDTree.pkl 3.2 MB loaded in 0.0s Area_5_office_34_KDTree.pkl 1.7 MB loaded in 0.0s Area_2_storage_5_KDTree.pkl 0.6 MB loaded in 0.0s Area_6_conferenceRoom_1_KDTree.pkl 2.0 MB loaded in 0.0s Area_3_hallway_5_KDTree.pkl 0.2 MB loaded in 0.0s Area_4_hallway_1_KDTree.pkl 2.6 MB loaded in 0.0s Area_4_office_5_KDTree.pkl 1.8 MB loaded in 0.0s Area_1_hallway_2_KDTree.pkl 1.0 MB loaded in 0.0s Area_2_office_13_KDTree.pkl 1.1 MB loaded in 0.0s Area_3_storage_2_KDTree.pkl 0.2 MB loaded in 0.0s Area_4_hallway_11_KDTree.pkl 1.4 MB loaded in 0.0s Area_2_office_3_KDTree.pkl 1.1 MB loaded in 0.0s Area_6_office_7_KDTree.pkl 1.5 MB loaded in 0.0s Area_5_office_10_KDTree.pkl 1.5 MB loaded in 0.0s Area_5_storage_2_KDTree.pkl 1.6 MB loaded in 0.0s Area_1_office_6_KDTree.pkl 1.2 MB loaded in 0.0s Area_5_office_8_KDTree.pkl 1.7 MB loaded in 0.0s Area_5_office_30_KDTree.pkl 0.9 MB loaded in 0.0s Area_5_hallway_7_KDTree.pkl 2.2 MB loaded in 0.0s Area_4_hallway_8_KDTree.pkl 0.2 MB loaded in 0.0s Area_4_hallway_6_KDTree.pkl 0.3 MB loaded in 0.0s Area_5_office_31_KDTree.pkl 1.6 MB loaded in 0.0s Area_4_office_7_KDTree.pkl 1.3 MB loaded in 0.0s Area_2_hallway_5_KDTree.pkl 5.5 MB loaded in 0.0s Area_6_hallway_1_KDTree.pkl 6.0 MB loaded in 0.0s Area_5_office_33_KDTree.pkl 1.6 MB loaded in 0.0s Area_2_storage_2_KDTree.pkl 0.7 MB loaded in 0.0s Area_3_office_10_KDTree.pkl 1.1 MB loaded in 0.0s Area_5_office_28_KDTree.pkl 1.5 MB loaded in 0.0s Area_5_office_1_KDTree.pkl 1.6 MB loaded in 0.0s Area_1_office_21_KDTree.pkl 1.7 MB loaded in 0.0s Area_3_office_1_KDTree.pkl 1.5 MB loaded in 0.0s Area_3_hallway_1_KDTree.pkl 1.8 MB loaded in 0.0s Area_1_office_26_KDTree.pkl 1.6 MB loaded in 0.0s Area_5_office_24_KDTree.pkl 3.1 MB loaded in 0.0s Area_5_storage_3_KDTree.pkl 0.4 MB loaded in 0.0s Area_4_WC_4_KDTree.pkl 1.2 MB loaded in 0.0s Area_5_hallway_8_KDTree.pkl 0.9 MB loaded in 0.0s Area_3_office_2_KDTree.pkl 1.5 MB loaded in 0.0s Area_1_copyRoom_1_KDTree.pkl 1.0 MB loaded in 0.0s Area_5_office_22_KDTree.pkl 1.5 MB loaded in 0.0s Area_5_office_15_KDTree.pkl 2.5 MB loaded in 0.0s Area_6_openspace_1_KDTree.pkl 3.7 MB loaded in 0.0s Area_2_office_8_KDTree.pkl 1.1 MB loaded in 0.0s Area_2_office_7_KDTree.pkl 0.9 MB loaded in 0.0s Area_5_conferenceRoom_2_KDTree.pkl 3.7 MB loaded in 0.0s Area_2_office_5_KDTree.pkl 2.2 MB loaded in 0.0s Area_4_conferenceRoom_3_KDTree.pkl 1.9 MB loaded in 0.0s Area_4_office_1_KDTree.pkl 1.1 MB loaded in 0.0s Area_6_office_6_KDTree.pkl 1.5 MB loaded in 0.0s Area_4_office_22_KDTree.pkl 1.0 MB loaded in 0.0s Area_2_office_2_KDTree.pkl 0.9 MB loaded in 0.0s Area_1_office_3_KDTree.pkl 1.6 MB loaded in 0.0s Area_1_office_13_KDTree.pkl 1.3 MB loaded in 0.0s Area_4_office_12_KDTree.pkl 1.2 MB loaded in 0.0s Area_6_office_37_KDTree.pkl 1.5 MB loaded in 0.0s Area_6_lounge_1_KDTree.pkl 2.6 MB loaded in 0.0s Area_6_office_13_KDTree.pkl 0.9 MB loaded in 0.0s Area_3_hallway_2_KDTree.pkl 2.2 MB loaded in 0.0s Area_2_office_10_KDTree.pkl 1.0 MB loaded in 0.0s Area_2_storage_7_KDTree.pkl 0.7 MB loaded in 0.0s Area_5_office_3_KDTree.pkl 1.5 MB loaded in 0.0s Area_1_office_20_KDTree.pkl 1.7 MB loaded in 0.0s Area_3_office_9_KDTree.pkl 1.2 MB loaded in 0.0s Area_5_office_35_KDTree.pkl 1.7 MB loaded in 0.0s Area_2_hallway_1_KDTree.pkl 0.5 MB loaded in 0.0s Area_5_office_27_KDTree.pkl 1.4 MB loaded in 0.0s Area_4_conferenceRoom_1_KDTree.pkl 2.1 MB loaded in 0.0s Area_2_office_4_KDTree.pkl 2.2 MB loaded in 0.0s Area_1_hallway_1_KDTree.pkl 0.6 MB loaded in 0.0s Area_2_office_11_KDTree.pkl 1.2 MB loaded in 0.0s Area_5_office_21_KDTree.pkl 3.8 MB loaded in 0.0s Area_5_office_32_KDTree.pkl 1.5 MB loaded in 0.0s Area_2_storage_9_KDTree.pkl 0.5 MB loaded in 0.0s Area_5_office_37_KDTree.pkl 5.3 MB loaded in 0.0s Area_2_office_1_KDTree.pkl 0.8 MB loaded in 0.0s Area_4_hallway_10_KDTree.pkl 2.4 MB loaded in 0.0s Area_6_office_28_KDTree.pkl 1.5 MB loaded in 0.0s Area_2_storage_6_KDTree.pkl 0.5 MB loaded in 0.0s Area_1_conferenceRoom_1_KDTree.pkl 2.2 MB loaded in 0.0s Area_4_storage_2_KDTree.pkl 0.6 MB loaded in 0.0s Area_2_storage_3_KDTree.pkl 0.7 MB loaded in 0.0s Area_1_hallway_3_KDTree.pkl 0.7 MB loaded in 0.0s Area_1_office_5_KDTree.pkl 1.2 MB loaded in 0.0s Area_6_hallway_3_KDTree.pkl 0.6 MB loaded in 0.0s Area_1_hallway_7_KDTree.pkl 6.2 MB loaded in 0.0s Area_3_office_3_KDTree.pkl 1.6 MB loaded in 0.0s Area_6_office_18_KDTree.pkl 1.0 MB loaded in 0.0s Area_3_lounge_2_KDTree.pkl 3.7 MB loaded in 0.0s Area_6_office_17_KDTree.pkl 1.0 MB loaded in 0.0s Area_5_hallway_9_KDTree.pkl 2.2 MB loaded in 0.0s Area_1_office_29_KDTree.pkl 1.0 MB loaded in 0.0s Area_4_storage_4_KDTree.pkl 0.6 MB loaded in 0.0s Area_6_hallway_6_KDTree.pkl 1.8 MB loaded in 0.0s Area_5_hallway_15_KDTree.pkl 2.4 MB loaded in 0.0s Area_5_office_29_KDTree.pkl 3.1 MB loaded in 0.0s Area_5_conferenceRoom_3_KDTree.pkl 2.9 MB loaded in 0.0s Area_5_storage_1_KDTree.pkl 1.3 MB loaded in 0.0s Area_4_lobby_1_KDTree.pkl 7.6 MB loaded in 0.0s Area_2_hallway_9_KDTree.pkl 0.7 MB loaded in 0.0s Area_5_WC_2_KDTree.pkl 1.2 MB loaded in 0.0s Area_5_office_18_KDTree.pkl 2.4 MB loaded in 0.0s Area_5_office_11_KDTree.pkl 1.7 MB loaded in 0.0s Area_4_lobby_2_KDTree.pkl 3.9 MB loaded in 0.0s Area_6_office_35_KDTree.pkl 1.5 MB loaded in 0.0s Area_2_office_6_KDTree.pkl 0.9 MB loaded in 0.0s Area_6_office_14_KDTree.pkl 0.9 MB loaded in 0.0s Area_5_office_16_KDTree.pkl 1.4 MB loaded in 0.0s Area_2_storage_1_KDTree.pkl 0.4 MB loaded in 0.0s Area_1_conferenceRoom_2_KDTree.pkl 3.0 MB loaded in 0.0s Area_6_hallway_2_KDTree.pkl 5.9 MB loaded in 0.0s Area_1_office_9_KDTree.pkl 1.6 MB loaded in 0.0s Area_6_office_5_KDTree.pkl 1.6 MB loaded in 0.0s Area_5_office_5_KDTree.pkl 1.4 MB loaded in 0.0s Area_2_hallway_10_KDTree.pkl 4.7 MB loaded in 0.0s Area_3_hallway_3_KDTree.pkl 0.3 MB loaded in 0.0s Area_4_office_13_KDTree.pkl 1.3 MB loaded in 0.0s Area_5_office_7_KDTree.pkl 1.6 MB loaded in 0.0s Area_1_office_7_KDTree.pkl 1.2 MB loaded in 0.0s Area_2_storage_4_KDTree.pkl 0.8 MB loaded in 0.0s Area_1_office_14_KDTree.pkl 1.3 MB loaded in 0.0s Area_6_office_30_KDTree.pkl 1.5 MB loaded in 0.0s Area_4_hallway_7_KDTree.pkl 2.9 MB loaded in 0.0s Area_2_hallway_6_KDTree.pkl 0.5 MB loaded in 0.0s Area_3_hallway_4_KDTree.pkl 2.9 MB loaded in 0.0s Area_1_WC_1_KDTree.pkl 2.1 MB loaded in 0.0s Area_5_office_25_KDTree.pkl 1.2 MB loaded in 0.0s Area_2_hallway_7_KDTree.pkl 0.5 MB loaded in 0.0s Area_5_hallway_14_KDTree.pkl 1.4 MB loaded in 0.0s Area_1_office_18_KDTree.pkl 1.5 MB loaded in 0.0s Area_3_office_8_KDTree.pkl 2.0 MB loaded in 0.0s Area_2_WC_1_KDTree.pkl 1.1 MB loaded in 0.0s Area_4_hallway_3_KDTree.pkl 3.9 MB loaded in 0.0s Area_5_storage_4_KDTree.pkl 1.4 MB loaded in 0.0s Area_2_hallway_12_KDTree.pkl 2.3 MB loaded in 0.0s Area_2_hallway_2_KDTree.pkl 2.1 MB loaded in 0.0s Area_4_office_16_KDTree.pkl 4.1 MB loaded in 0.0s Area_6_office_33_KDTree.pkl 1.5 MB loaded in 0.0s Area_5_office_23_KDTree.pkl 1.7 MB loaded in 0.0s Area_1_office_27_KDTree.pkl 1.6 MB loaded in 0.0s Area_2_hallway_4_KDTree.pkl 0.7 MB loaded in 0.0s Area_6_office_3_KDTree.pkl 1.5 MB loaded in 0.0s Area_3_hallway_6_KDTree.pkl 0.4 MB loaded in 0.0s Area_3_office_6_KDTree.pkl 1.5 MB loaded in 0.0s Area_5_hallway_10_KDTree.pkl 2.2 MB loaded in 0.0s Area_6_office_24_KDTree.pkl 1.5 MB loaded in 0.0s Area_5_office_14_KDTree.pkl 2.3 MB loaded in 0.0s Area_1_office_23_KDTree.pkl 1.6 MB loaded in 0.0s Area_5_pantry_1_KDTree.pkl 1.3 MB loaded in 0.0s Area_1_office_11_KDTree.pkl 1.6 MB loaded in 0.0s Area_4_office_10_KDTree.pkl 2.1 MB loaded in 0.0s Area_5_hallway_11_KDTree.pkl 1.4 MB loaded in 0.0s Area_6_office_22_KDTree.pkl 0.9 MB loaded in 0.0s Area_6_office_2_KDTree.pkl 1.5 MB loaded in 0.0s Area_6_office_26_KDTree.pkl 1.6 MB loaded in 0.0s Area_1_office_8_KDTree.pkl 1.6 MB loaded in 0.0s Area_2_storage_8_KDTree.pkl 0.5 MB loaded in 0.0s Area_2_auditorium_1_KDTree.pkl 13.8 MB loaded in 0.0s Area_5_office_13_KDTree.pkl 2.0 MB loaded in 0.0s Area_4_hallway_9_KDTree.pkl 0.7 MB loaded in 0.0s Area_5_hallway_3_KDTree.pkl 2.3 MB loaded in 0.0s Area_4_conferenceRoom_2_KDTree.pkl 3.1 MB loaded in 0.0s Area_2_office_12_KDTree.pkl 1.2 MB loaded in 0.0s Area_6_office_23_KDTree.pkl 1.5 MB loaded in 0.0s Area_4_office_6_KDTree.pkl 1.3 MB loaded in 0.0s Area_4_office_3_KDTree.pkl 1.1 MB loaded in 0.0s Area_5_hallway_13_KDTree.pkl 2.4 MB loaded in 0.0s Area_5_hallway_12_KDTree.pkl 1.4 MB loaded in 0.0s Area_1_hallway_4_KDTree.pkl 0.7 MB loaded in 0.0s Area_3_WC_2_KDTree.pkl 2.1 MB loaded in 0.0s Area_3_lounge_1_KDTree.pkl 2.0 MB loaded in 0.0s Area_4_WC_1_KDTree.pkl 0.5 MB loaded in 0.0s Area_5_office_20_KDTree.pkl 1.0 MB loaded in 0.0s Area_6_pantry_1_KDTree.pkl 0.9 MB loaded in 0.0s Area_6_office_15_KDTree.pkl 1.0 MB loaded in 0.0s Area_1_office_10_KDTree.pkl 1.5 MB loaded in 0.0s Area_5_hallway_4_KDTree.pkl 1.9 MB loaded in 0.0s Area_2_WC_2_KDTree.pkl 1.6 MB loaded in 0.0s Area_1_hallway_6_KDTree.pkl 7.1 MB loaded in 0.0s Area_4_storage_3_KDTree.pkl 0.7 MB loaded in 0.0s Area_4_office_4_KDTree.pkl 1.6 MB loaded in 0.0s Area_1_office_24_KDTree.pkl 1.6 MB loaded in 0.0s Area_5_hallway_2_KDTree.pkl 8.1 MB loaded in 0.0s Area_6_office_19_KDTree.pkl 1.0 MB loaded in 0.0s Area_6_office_16_KDTree.pkl 1.2 MB loaded in 0.0s Area_4_office_8_KDTree.pkl 1.2 MB loaded in 0.0s Area_6_office_34_KDTree.pkl 1.4 MB loaded in 0.0s Area_1_office_2_KDTree.pkl 1.7 MB loaded in 0.0s Area_6_office_29_KDTree.pkl 1.6 MB loaded in 0.0s Area_6_office_11_KDTree.pkl 1.1 MB loaded in 0.0s Area_6_office_20_KDTree.pkl 1.0 MB loaded in 0.0s Area_3_office_7_KDTree.pkl 1.7 MB loaded in 0.0s Area_1_office_1_KDTree.pkl 1.7 MB loaded in 0.0s Area_5_office_4_KDTree.pkl 1.6 MB loaded in 0.0s Area_1_hallway_8_KDTree.pkl 2.3 MB loaded in 0.0s Area_4_office_15_KDTree.pkl 1.3 MB loaded in 0.0s Area_5_conferenceRoom_1_KDTree.pkl 1.9 MB loaded in 0.0s Area_6_hallway_5_KDTree.pkl 1.1 MB loaded in 0.0s Area_1_office_31_KDTree.pkl 7.4 MB loaded in 0.0s Area_5_office_6_KDTree.pkl 1.5 MB loaded in 0.0s Area_1_hallway_5_KDTree.pkl 0.9 MB loaded in 0.0s Area_5_office_17_KDTree.pkl 1.8 MB loaded in 0.0s Area_1_office_25_KDTree.pkl 1.5 MB loaded in 0.0s Area_4_office_17_KDTree.pkl 2.0 MB loaded in 0.0s Area_3_storage_1_KDTree.pkl 0.4 MB loaded in 0.0s Area_4_office_18_KDTree.pkl 1.5 MB loaded in 0.0s Area_5_hallway_1_KDTree.pkl 6.3 MB loaded in 0.0s Area_6_office_8_KDTree.pkl 2.9 MB loaded in 0.0s Area_4_hallway_2_KDTree.pkl 2.5 MB loaded in 0.0s Area_6_hallway_4_KDTree.pkl 0.6 MB loaded in 0.0s Area_5_office_42_KDTree.pkl 1.6 MB loaded in 0.0s Area_4_office_2_KDTree.pkl 1.0 MB loaded in 0.0s Area_6_office_12_KDTree.pkl 1.2 MB loaded in 0.0s Area_4_office_20_KDTree.pkl 1.8 MB loaded in 0.0s Area_4_office_11_KDTree.pkl 1.3 MB loaded in 0.0s Area_6_office_10_KDTree.pkl 1.5 MB loaded in 0.0s Area_1_office_17_KDTree.pkl 1.7 MB loaded in 0.0s Area_2_hallway_8_KDTree.pkl 7.6 MB loaded in 0.0s Area_1_office_30_KDTree.pkl 3.7 MB loaded in 0.0s Area_1_office_4_KDTree.pkl 1.2 MB loaded in 0.0s Area_5_office_38_KDTree.pkl 5.7 MB loaded in 0.0s Area_4_WC_2_KDTree.pkl 0.9 MB loaded in 0.0s Area_6_office_32_KDTree.pkl 1.6 MB loaded in 0.0s Area_6_office_25_KDTree.pkl 1.5 MB loaded in 0.0s Area_4_hallway_12_KDTree.pkl 1.8 MB loaded in 0.0s Area_2_office_14_KDTree.pkl 2.7 MB loaded in 0.0s Area_4_hallway_14_KDTree.pkl 2.1 MB loaded in 0.0s Area_1_office_22_KDTree.pkl 1.6 MB loaded in 0.0s Area_5_office_2_KDTree.pkl 1.7 MB loaded in 0.0s Area_6_copyRoom_1_KDTree.pkl 1.0 MB loaded in 0.0s Area_2_office_9_KDTree.pkl 1.0 MB loaded in 0.0s Area_1_office_16_KDTree.pkl 2.7 MB loaded in 0.0s Area_1_office_19_KDTree.pkl 1.6 MB loaded in 0.0s Area_5_office_9_KDTree.pkl 1.5 MB loaded in 0.0s Area_1_office_28_KDTree.pkl 1.4 MB loaded in 0.0s Area_5_office_39_KDTree.pkl 1.8 MB loaded in 0.0s Area_2_auditorium_2_KDTree.pkl 18.4 MB loaded in 0.0s Area_3_WC_1_KDTree.pkl 2.0 MB loaded in 0.0s Area_6_office_36_KDTree.pkl 1.6 MB loaded in 0.0s Area_6_office_27_KDTree.pkl 1.5 MB loaded in 0.0s Area_3_office_5_KDTree.pkl 1.5 MB loaded in 0.0s Area_4_storage_1_KDTree.pkl 0.3 MB loaded in 0.0s Area_5_hallway_5_KDTree.pkl 5.1 MB loaded in 0.0s Area_1_pantry_1_KDTree.pkl 1.0 MB loaded in 0.0s Area_5_lobby_1_KDTree.pkl 2.0 MB loaded in 0.0s Area_2_hallway_3_KDTree.pkl 2.2 MB loaded in 0.0s Area_4_office_21_KDTree.pkl 1.8 MB loaded in 0.0s Area_3_office_4_KDTree.pkl 1.7 MB loaded in 0.0s Area_4_hallway_5_KDTree.pkl 0.2 MB loaded in 0.0s Area_2_hallway_11_KDTree.pkl 0.9 MB loaded in 0.0s Area_5_office_12_KDTree.pkl 1.6 MB loaded in 0.0s Area_5_office_26_KDTree.pkl 1.4 MB loaded in 0.0s Area_5_office_40_KDTree.pkl 4.9 MB loaded in 0.0s Area_4_WC_3_KDTree.pkl 0.4 MB loaded in 0.0s Area_6_office_31_KDTree.pkl 1.0 MB loaded in 0.0s Area_6_office_4_KDTree.pkl 1.4 MB loaded in 0.0s Area_3_conferenceRoom_1_KDTree.pkl 2.4 MB loaded in 0.0s Area_4_office_9_KDTree.pkl 2.0 MB loaded in 0.0s Area_2_conferenceRoom_1_KDTree.pkl 4.3 MB loaded in 0.0s Area_4_office_19_KDTree.pkl 1.3 MB loaded in 0.0s Area_1_office_15_KDTree.pkl 0.9 MB loaded in 0.0s Area_5_office_41_KDTree.pkl 2.7 MB loaded in 0.0s Area_6_office_1KDTree.pkl 1.5 MB loaded in 0.0s

_Preparing reprojected indices for testing Area_1_office_12 done in 0.0s Area_1_hallway_2 done in 0.0s Area_1_office_6 done in 0.0s Area_1_office_21 done in 0.0s Area_1_office_26 done in 0.0s Area_1_copyRoom_1 done in 0.0s Area_1_office_3 done in 0.0s Area_1_office_13 done in 0.0s Area_1_office_20 done in 0.0s Area_1_hallway_1 done in 0.0s Area_1_conferenceRoom_1 done in 0.0s Area_1_hallway_3 done in 0.0s Area_1_office_5 done in 0.0s Area_1_hallway_7 done in 0.0s Area_1_office_29 done in 0.0s Area_1_conferenceRoom_2 done in 0.0s Area_1_office_9 done in 0.0s Area_1_office_7 done in 0.0s Area_1_office_14 done in 0.0s Area_1_WC_1 done in 0.0s Area_1_office_18 done in 0.0s Area_1_office_27 done in 0.0s Area_1_office_23 done in 0.0s Area_1_office_11 done in 0.0s Area_1_office_8 done in 0.0s Area_1_hallway_4 done in 0.0s Area_1_office_10 done in 0.0s Area_1_hallway_6 done in 0.0s Area_1_office_24 done in 0.0s Area_1_office_2 done in 0.0s Area_1_office_1 done in 0.0s Area_1_hallway_8 done in 0.0s Area_1_office_31 done in 0.0s Area_1_hallway_5 done in 0.0s Area_1_office_25 done in 0.0s Area_1_office_17 done in 0.0s Area_1_office_30 done in 0.0s Area_1_office_4 done in 0.0s Area_1_office_22 done in 0.0s Area_1_office_16 done in 0.0s Area_1_office_19 done in 0.0s Area_1_office_28 done in 0.0s Area_1_pantry_1 done in 0.0s Area_1_office_15 done in 0.0s Initiating input pipelines WARNING:tensorflow:From /home/shji/Documents/RandLA-Net/RandLANet.py:265: softmax_cross_entropy_with_logits (from tensorflow.python.ops.nn_ops) is deprecated and will be removed in a future version. Instructions for updating:__

Future major versions of TensorFlow will allow gradients to flow into the labels input on backprop by default.

_See tf.nn.softmax_cross_entropy_with_logits_v2._

_/home/shji/anaconda3/envs/ranet/lib/python3.6/site-packages/tensorflow/python/ops/gradientsimpl.py:108: UserWarning: Converting sparse IndexedSlices to a dense Tensor of unknown shape. This may consume a large amount of memory. "Converting sparse IndexedSlices to a dense Tensor of unknown shape. " EPOCH 0 _2021-08-09 13:15:37.625090: E tensorflow/stream_executor/cuda/cuda_blas.cc:652] failed to run cuBLAS routine cublasSgemm_v2: CUBLAS_STATUS_EXECUTION_FAILED_

I'm curious as to why this error occurs. thank you.

[SuperKX]

@IMSHJI samequestion，did you solve it？thanks

[IMSHJI]

@SuperKX No.. Data is not generated on rtx3070, so I am using gtx1660ti.

[SuperKX]

@IMSHJI me too, haha. I have tried on rtx3080. but people said that rtx30 sieries don't support CUDA10, and higher CUDA version don't support tensorflow1.14. higher tensorflow version don't work. hhhh

[youssef962]

what did you use as OS? I'm trying to install ubuntu 16.04 but it's EOL version and don't want to be installed! Also what is the solution to run on rtx30 series?

[hdmapserver]

@IMSHJI me too, haha. I have tried on rtx3080. but people said that rtx30 sieries don't support CUDA10, and higher CUDA version don't support tensorflow1.14. higher tensorflow version don't work. hhhh

I learn from this link: https://blog.csdn.net/wu496963386/article/details/109583045?utm_source=app&app_version=4.8.1

[hdmapserver]

After struggling for almost a week, I successfully run RandLA-Net on my computer by docker, and my environment is RTX3070 + cuda11.1 + cudnn8.0.4 + python3.6.9 + TensorFlow1.15

[hdmapserver]

my docker image downloaded from this link: https://hub.docker.com/r/nvidia/cuda/tags?page=1&name=11.1.1-cudnn8-devel-ubuntu18.04

my bash record are as follows

docker pull nvidia/cuda:11.1.1-cudnn8-devel-ubuntu18.04 / you need to change the share filepath and ImageID below (my imageID is 5a214d77f5d7)/ docker run -it -v /home/hdmap/qbdata:/home/hdmap/qbdata --gpus all --shm-size 60G -u root --name randla-net 5a214d77f5d7 /bin/bash / then you will enter a container, I ran these commands and finally succeeded/ apt-get update apt-get install python3.6 apt install python3 apt install python3-pip pip3 install nvidia-pyindex apt install --fix-missing python3-pip python3 -m ensurepip pip3 install --upgrade pip pip install nvidia-pyindex pip install nvidia-tensorflow cd /home/hdmap/qbdata/data/RandLA-Net pip install -r helper_requirements.txt apt install libgl1-mesa-glx sh compile_op.sh / then edit the dataset path in utils/data_prepare_s3dis.py/ python utils/data_prepare_s3dis.py / then set smaller batch_size and val_batch_size in helper_tool.py/ sh jobs_6_fold_cv_s3dis.sh

[SC-shendazt]

After struggling for almost a week, I successfully run RandLA-Net on my computer by docker, and my environment is RTX3070 + cuda11.1 + cudnn8.0.4 + python3.6.9 + TensorFlow1.15

@hdmapserver 您好，我也下载了dockerrandlanet，但是一运行就报错如下： EPOCH 0 2021-08-09 13:15:37.625090: E tensorflow/stream_executor/cuda/cuda_blas.cc:652] failed to run cuBLAS routine cublasSgemm_v2: CUBLAS_STATUS_EXECUTION_FAILED 请问是我宿主机cuda的版本问题吗，但是我是下载的别人装好的randla镜像啊，他里面都包含这个代码的呢

[szyilmaz]

Here a solution of mine for windows (wsl2-ubuntu) 1)cuda toolkit (windows) https://developer.nvidia.com/cuda-11.1.0-download-archive?target_os=Windows&target_arch=x86_64&target_version=10&target_type=exelocal 2)setup Ubuntu wsl 22.04.2 3)enable gpu on wsl https://ubuntu.com/tutorials/enabling-gpu-acceleration-on-ubuntu-on-wsl2-with-the-nvidia-cuda-platform#2-install-the-appropriate-windows-vgpu-driver-for-wsl 4)miniconda sudo apt-get update cd /tmp apt-get install wget wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh sha256sum Miniconda3-latest-Linux-x86_64.sh bash Miniconda3-latest-Linux-x86_64.sh 5) How To Install TensorFlow 1.15 for NVIDIA RTX30 GPUs (without docker or CUDA install) https://www.pugetsystems.com/labs/hpc/how-to-install-tensorflow-1-15-for-nvidia-rtx30-gpus-without-docker-or-cuda-install-2005/ 6)git clone --depth=1 https://github.com/QingyongHu/RandLA-Net && cd RandLA-Net 7)compile_op.sh içindeki adımları manuel uygula, hata verek kütüphaneleri helper_requirements'ten kur 8)python main_Semantic3D.py --mode train --gpu 0