Closed lyh1028 closed 8 months ago
[Taichi] version 1.8.0, llvm 15.0.4, commit 52b24f3e, linux, python 3.8.18 [W 03/03/24 17:05:31.950 2179169] [cuda_driver.cpp:load_lib@36] libcuda.so lib not found. [W 03/03/24 17:05:31.950 2179169] [misc.py:adaptive_arch_select@758] Arch=[<Arch.cuda: 3>] is not supported, falling back to CPU [Taichi] Starting on arch=x64 Loading 161 train images ... 100%|██████████████████████████████████████████████████████████████████████████████████████████████████| 161/161 [00:05<00:00, 31.34it/s] Loading 24 test images ... 100%|████████████████████████████████████████████████████████████████████████████████████████████████████| 24/24 [00:00<00:00, 37.24it/s] Hash Encoder: base_res=16 max_res=4096 hash_level=16 feat_per_level=2 per_level_scale=0.36967849629863747 total_hash_size=6299960 Failed to import apex FusedAdam, use torch Adam instead. ./scripts/train_360_v2_garden.sh: line 11: 2179169 Bus error (core dumped) python3 train.py --root_dir $ROOT_DIR/garden --dataset_name colmap --exp_name garden --downsample $DOWNSAMPLE --scale 8.0 --batch_size 4096
I have tried reducing batch_size but it didn't work. I wonder why there's waring "libcuda.so lib not found."
Have solved. re-install cuda toolkit and find libcuda.so path, and tried to add them one by one to $LD_LIBRARY_PATH
[Taichi] version 1.8.0, llvm 15.0.4, commit 52b24f3e, linux, python 3.8.18 [W 03/03/24 17:05:31.950 2179169] [cuda_driver.cpp:load_lib@36] libcuda.so lib not found. [W 03/03/24 17:05:31.950 2179169] [misc.py:adaptive_arch_select@758] Arch=[<Arch.cuda: 3>] is not supported, falling back to CPU [Taichi] Starting on arch=x64 Loading 161 train images ... 100%|██████████████████████████████████████████████████████████████████████████████████████████████████| 161/161 [00:05<00:00, 31.34it/s] Loading 24 test images ... 100%|████████████████████████████████████████████████████████████████████████████████████████████████████| 24/24 [00:00<00:00, 37.24it/s] Hash Encoder: base_res=16 max_res=4096 hash_level=16 feat_per_level=2 per_level_scale=0.36967849629863747 total_hash_size=6299960 Failed to import apex FusedAdam, use torch Adam instead. ./scripts/train_360_v2_garden.sh: line 11: 2179169 Bus error (core dumped) python3 train.py --root_dir $ROOT_DIR/garden --dataset_name colmap --exp_name garden --downsample $DOWNSAMPLE --scale 8.0 --batch_size 4096
I have tried reducing batch_size but it didn't work. I wonder why there's waring "libcuda.so lib not found."