search

xxlong0 / Wonder3D

Single Image to 3D using Cross-Domain Diffusion for 3D Generation
https://www.xxlong.site/Wonder3D/
GNU Affero General Public License v3.0
4.83k stars 389 forks source link

AttributeError: 'NoneType' object has no attribute 'ContractionType' #104

Open caishuo-C opened 11 months ago

caishuo-C commented 11 months ago

Prior to this, I had successfully run through several 3d mesh generation using Instant-NSR.

But recently when I tried to enter the same command, I got the following (error) :

command： cd ./instant-nsr-pl python launch.py --config configs/neuralangelo-ortho-wmask.yaml --gpu 5 --train dataset.root_dir=../outputs/cropsize-192-cfg3.0/ dataset.scene=stewed_meatball

Terminal Output：

Global seed set to 42                                                                                                                                                                                          
Using finite difference to compute gradients with eps=progressive                                                                                                                                              
Using 16bit None Automatic Mixed Precision (AMP)                                                                                                                                                               
GPU available: True (cuda), used: True                                                                                                                                                                         
TPU available: False, using: 0 TPU cores                                                                                                                                                                       
IPU available: False, using: 0 IPUs                                                                                                                                                                            
HPU available: False, using: 0 HPUs                                                                                                                                                                              09:09:39 
`Trainer(limit_train_batches=1.0)` was configured so 100% of the batches per epoch will be used..                                                                                                              
[rank: 0] Global seed set to 42                                                                        
Initializing distributed: GLOBAL_RANK: 0, MEMBER: 1/1                                                  
----------------------------------------------------------------------------------------------------                                                                                                           
distributed_backend=nccl                                                                               
All distributed processes registered. Starting with 1 processes                                                                                                                                                
----------------------------------------------------------------------------------------------------                                                                                                           

You are using a CUDA device ('NVIDIA GeForce RTX 3090') that has Tensor Cores. To properly utilize them, you should set `torch.set_float32_matmul_precision('medium' | 'high')` which will trade-off precision 
for performance. For more details, read https://pytorch.org/docs/stable/generated/torch.set_float32_matmul_precision.html#torch.set_float32_matmul_precision
../outputs/cropsize-192-cfg3.0/stewed_meatball                                                         
(1024, 1024, 3)                                    
the loaded normals are defined in the system of front view                                             
../outputs/cropsize-192-cfg3.0/stewed_meatball                                                         
(1024, 1024, 3)

LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [5]                                                              

  | Name  | Type             | Params                                                                  
-------------------------------------------                                                            
0 | cos   | CosineSimilarity | 0                                                                       
1 | model | NeuSModel        | 7.7 M                                                                   
-------------------------------------------                                                            
7.7 M     Trainable params                         
0         Non-trainable params                                                                         
7.7 M     Total params                             
15.371    Total estimated model params size (MB)                                                       
Epoch 0: : 0it [00:00, ?it/s]Update finite_difference_eps to 0.027204705103003882

NerfAcc: No CUDA toolkit found. NerfAcc will be disabled.

Traceback (most recent call last):                                                                     
  File "launch.py", line 125, in <module>                                                              
    main()                                         
  File "launch.py", line 114, in main                                                                  
    trainer.fit(system, datamodule=dm)                                                                 
  File "/data/caishuo/miniconda3/envs/wonder3d/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 608, in fit
  call._call_and_handle_interrupt(                                                                                                                                                                           
  File "/data/caishuo/miniconda3/envs/wonder3d/lib/python3.8/site-packages/pytorch_lightning/trainer/call.py", line 36, in _call_and_handle_interrupt                                                          
    return trainer.strategy.launcher.launch(trainer_fn, *args, trainer=trainer, **kwargs)                                                                                                                      
  File "/data/caishuo/miniconda3/envs/wonder3d/lib/python3.8/site-packages/pytorch_lightning/strategies/launchers/subprocess_script.py", line 88, in launch                                                    
    return function(*args, **kwargs)                                                                   
  File "/data/caishuo/miniconda3/envs/wonder3d/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 650, in _fit_impl                                                                       
    self._run(model, ckpt_path=self.ckpt_path)                                                         
  File "/data/caishuo/miniconda3/envs/wonder3d/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 1112, in _run                                                                           
    results = self._run_stage()                                                                        
  File "/data/caishuo/miniconda3/envs/wonder3d/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 1191, in _run_stage                                                                     
    self._run_train()                              
  File "/data/caishuo/miniconda3/envs/wonder3d/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 1214, in _run_train                                                                     
    self.fit_loop.run()                            
  File "/data/caishuo/miniconda3/envs/wonder3d/lib/python3.8/site-packages/pytorch_lightning/loops/loop.py", line 199, in run                                                                                  
    self.advance(*args, **kwargs)                                                                      
  File "/data/caishuo/miniconda3/envs/wonder3d/lib/python3.8/site-packages/pytorch_lightning/loops/fit_loop.py", line 267, in advance                                                                          
    self._outputs = self.epoch_loop.run(self._data_fetcher)                                            
  File "/data/caishuo/miniconda3/envs/wonder3d/lib/python3.8/site-packages/pytorch_lightning/loops/loop.py", line 199, in run                                                                                  
    self.advance(*args, **kwargs)                                                                      
  File "/data/caishuo/miniconda3/envs/wonder3d/lib/python3.8/site-packages/pytorch_lightning/loops/epoch/training_epoch_loop.py", line 204, in advance                                                         
    response = self.trainer._call_lightning_module_hook("on_train_batch_start", batch, batch_idx)                                                                                                              
  File "/data/caishuo/miniconda3/envs/wonder3d/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 1356, in _call_lightning_module_hook
   output = fn(*args, **kwargs)                                                                       
  File "/data/caishuo/3D_generation/Wonder3D/instant-nsr-pl/systems/base.py", line 57, in on_train_batch_start                                                                                                 
    update_module_step(self.model, self.current_epoch, self.global_step)                                                                                                                                       
  File "/data/caishuo/3D_generation/Wonder3D/instant-nsr-pl/systems/utils.py", line 351, in update_module_step                                                                                                 
    m.update_step(epoch, global_step)                                                                  
  File "/data/caishuo/3D_generation/Wonder3D/instant-nsr-pl/models/neus.py", line 111, in update_step                                                                                                          
    self.occupancy_grid.every_n_step(step=global_step, occ_eval_fn=occ_eval_fn, occ_thre=self.config.get('grid_prune_occ_thre', 0.01))                                                                         
  File "/data/caishuo/miniconda3/envs/wonder3d/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context                                                                          
    return func(*args, **kwargs)                                                                       
  File "/data/caishuo/miniconda3/envs/wonder3d/lib/python3.8/site-packages/nerfacc/grid.py", line 288, in every_n_step                                                                                         
    self._update(                                  
  File "/data/caishuo/miniconda3/envs/wonder3d/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context                                                                          
    return func(*args, **kwargs)                                                                       
  File "/data/caishuo/miniconda3/envs/wonder3d/lib/python3.8/site-packages/nerfacc/grid.py", line 239, in _update                                                                                              
    x = contract_inv(                              
  File "/data/caishuo/miniconda3/envs/wonder3d/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context                                                                          
    return func(*args, **kwargs)                                                                       
  File "/data/caishuo/miniconda3/envs/wonder3d/lib/python3.8/site-packages/nerfacc/contraction.py", line 101, in contract_inv                                                                                  
    ctype = type.to_cpp_version()                                                                      
  File "/data/caishuo/miniconda3/envs/wonder3d/lib/python3.8/site-packages/nerfacc/contraction.py", line 62, in to_cpp_version                                                                                 
    return _C.ContractionTypeGetter(self.value)                                                        
  File "/data/caishuo/miniconda3/envs/wonder3d/lib/python3.8/site-packages/nerfacc/cuda/__init__.py", line 13, in call_cuda                                                                                    
    return getattr(_C, name)(*args, **kwargs)                                                          
AttributeError: 'NoneType' object has no attribute 'ContractionType'

Here I noticed that there seems to be a problem with nerfacc (NerfAcc: No CUDA toolkit found. NerfAcc will be disabled.), how to solve this problem?

env： torch 1.13.1+cu117 cuda 11.8 nerfacc 0.3.3 python 3.8.18

haodong2000 commented 10 months ago

Hi, did you solve this bug? I met this error, either.

caishuo-C commented 10 months ago

Hi, did you solve this bug? I met this error, either.

I've fixed the bug. The reason is that CUDA did not load successfully, you can check your environment from this perspective, if you are using a zsh terminal, check ~/.zshrc.

haodong2000 commented 10 months ago

thx! I fixed it!

flamehaze1115 commented 10 months ago

Hi, did you solve this bug? I met this error, either.

I've fixed the bug. The reason is that CUDA did not load successfully, you can check your environment from this perspective, if you are using a zsh terminal, check ~/.zshrc.

Thanks for your help!