Hello,
I followed the ReadMe, creating a conda environment, activating it and running the demo with hero_model and vdr dataset according to the section "Setup" and "Running out of the box!".
However it did not work but having an error in the end (RuntimeError: CUDA error: CUBLAS_STATUS_NOT_SUPPORTED when calling cublasSgemmStridedBatched( handle, opa, opb, m, n, k, &alpha, a, lda, stridea, b, ldb, strideb, &beta, c, ldc, stridec, num_batches)).
Could you please help me figuring it out, where I did wrong?
I have checked, pytorch version 1.10.0, CUDA version 11.3. My GPU is NVIDIA GeForce RTX 3080 Ti with sm86.
################################################################################
####################### VDR Dataset, number of scans: 2 ########################
################################################################################
WARNING - 2024-07-30 10:26:22,099 - warnings - /home/aime/miniconda3/envs/simplerecon/lib/python3.9/site-packages/timm/models/_factory.py:117: UserWarning: Mapping deprecated model name tf_efficientnetv2_s_in21ft1k to current tf_efficientnetv2_s.in21k_ft_in1k.
model = create_fn(
INFO - 2024-07-30 10:26:22,268 - _builder - Loading pretrained weights from Hugging Face hub (timm/tf_efficientnetv2_s.in21k_ft_in1k)
INFO - 2024-07-30 10:26:22,488 - _hub - [timm/tf_efficientnetv2_s.in21k_ft_in1k] Safe alternative available for 'pytorch_model.bin' (as 'model.safetensors'). Loading weights using safetensors.
WARNING - 2024-07-30 10:26:22,574 - _builder - Unexpected keys (bn2.bias, bn2.num_batches_tracked, bn2.running_mean, bn2.running_var, bn2.weight, classifier.bias, classifier.weight, conv_head.weight) found while loading pretrained weights. This may be expected if model is being adapted.
################################################################################
########################## Using FeatureVolumeManager ##########################
Number of source views: 7
Using all metadata.
Number of channels: [202, 128, 128, 1]
################################################################################
################################################################################
########################## Using FeatureVolumeManager ##########################
Number of source views: 7
Using all metadata.
Number of channels: [202, 128, 128, 1]
################################################################################
################################################################################
######################## Using FastFeatureVolumeManager ########################
Number of source views: 7
Using all metadata.
Number of channels: [202, 128, 128, 1]
################################################################################
0%| | 0/562 [00:04<?, ?it/s]
0%| | 0/2 [00:04<?, ?it/s]
Traceback (most recent call last):
File "/home/aime/bliu_workspace/src/simplerecon/test.py", line 473, in
main(opts)
File "/home/aime/bliu_workspace/src/simplerecon/test.py", line 270, in main
outputs = model(
File "/home/aime/miniconda3/envs/simplerecon/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "/home/aime/bliu_workspace/src/simplerecon/experiment_modules/depth_model.py", line 328, in forward
src_cam_T_cur_cam = src_cam_T_world @ cur_world_T_cam.unsqueeze(1)
RuntimeError: CUDA error: CUBLAS_STATUS_NOT_SUPPORTED when calling cublasSgemmStridedBatched( handle, opa, opb, m, n, k, &alpha, a, lda, stridea, b, ldb, strideb, &beta, c, ldc, stridec, num_batches)
Hello, I followed the ReadMe, creating a conda environment, activating it and running the demo with hero_model and vdr dataset according to the section "Setup" and "Running out of the box!". However it did not work but having an error in the end (RuntimeError: CUDA error: CUBLAS_STATUS_NOT_SUPPORTED when calling
cublasSgemmStridedBatched( handle, opa, opb, m, n, k, &alpha, a, lda, stridea, b, ldb, strideb, &beta, c, ldc, stridec, num_batches)
). Could you please help me figuring it out, where I did wrong? I have checked, pytorch version 1.10.0, CUDA version 11.3. My GPU is NVIDIA GeForce RTX 3080 Ti with sm86.Thank you!
Below you can find the loggings in my terminal.
/src/simplerecon(main)$ CUDA_VISIBLE_DEVICES=0 python test.py --name HERO_MODEL \
###############################################################
################################################################################ ####################### VDR Dataset, number of scans: 2 ######################## ################################################################################
################################################################################ ######################### Running fusion! Using open3d ######################### Output directory: OUTPUT_PATH/HERO_MODEL/vdr/dense/meshes/0.04_3.0_open3d_color ################################################################################
################################################################################ ############################### Saving quick viz.############################### #######Output directory: OUTPUT_PATH/HERO_MODEL/vdr/dense/viz/quick_viz ######## ################################################################################
WARNING - 2024-07-30 10:26:22,099 - warnings - /home/aime/miniconda3/envs/simplerecon/lib/python3.9/site-packages/timm/models/_factory.py:117: UserWarning: Mapping deprecated model name tf_efficientnetv2_s_in21ft1k to current tf_efficientnetv2_s.in21k_ft_in1k. model = create_fn(
INFO - 2024-07-30 10:26:22,268 - _builder - Loading pretrained weights from Hugging Face hub (timm/tf_efficientnetv2_s.in21k_ft_in1k) INFO - 2024-07-30 10:26:22,488 - _hub - [timm/tf_efficientnetv2_s.in21k_ft_in1k] Safe alternative available for 'pytorch_model.bin' (as 'model.safetensors'). Loading weights using safetensors. WARNING - 2024-07-30 10:26:22,574 - _builder - Unexpected keys (bn2.bias, bn2.num_batches_tracked, bn2.running_mean, bn2.running_var, bn2.weight, classifier.bias, classifier.weight, conv_head.weight) found while loading pretrained weights. This may be expected if model is being adapted. ################################################################################ ########################## Using FeatureVolumeManager ########################## Number of source views: 7
Using all metadata.
Number of channels: [202, 128, 128, 1]
################################################################################
################################################################################ ########################## Using FeatureVolumeManager ########################## Number of source views: 7
Using all metadata.
Number of channels: [202, 128, 128, 1]
################################################################################
################################################################################ ######################## Using FastFeatureVolumeManager ######################## Number of source views: 7
Using all metadata.
Number of channels: [202, 128, 128, 1]
################################################################################
0%| | 0/562 [00:04<?, ?it/s] 0%| | 0/2 [00:04<?, ?it/s] Traceback (most recent call last): File "/home/aime/bliu_workspace/src/simplerecon/test.py", line 473, in
main(opts)
File "/home/aime/bliu_workspace/src/simplerecon/test.py", line 270, in main
outputs = model(
File "/home/aime/miniconda3/envs/simplerecon/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "/home/aime/bliu_workspace/src/simplerecon/experiment_modules/depth_model.py", line 328, in forward
src_cam_T_cur_cam = src_cam_T_world @ cur_world_T_cam.unsqueeze(1)
RuntimeError: CUDA error: CUBLAS_STATUS_NOT_SUPPORTED when calling
cublasSgemmStridedBatched( handle, opa, opb, m, n, k, &alpha, a, lda, stridea, b, ldb, strideb, &beta, c, ldc, stridec, num_batches)