jzr99
commented
1 month ago Hi,
I thought there is some compatibility issue with the environment.
- First of all, your pytorch version is 1.10 which should be companied with torchvision 0.11 according to https://pypi.org/project/torchvision/ (but we recommend you use the same torch (1.12.0) and torchvision (0.13.0) cu113 with us.)
- Second, which nerfacc version do you use? we use 0.5.3. Since there is no prebuild nerfacc for torch 1.10.0 CUDA 111 (even I do not know if nerfacc is compatiable with torch 1.10.0 ) , could you please build it from source following the instruction here Also, there are some prebuild nerfacc to use if you use exactly same torch version (1.12.0+cu113) and torchvision version (0.13.0+cu113) with us.
Hello, can I ask a question during training, when I run train.py, the system raise an error: IndexError: Dimension out of range (expected to be in range of [-1, 0], but got 1) The error message is shown below: ''' Error executing job with overrides: [] Traceback (most recent call last): File "train.py", line 47, in main trainer.fit(model, trainset, validset) File "/home/haolin/anaconda3/envs/multiply/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 609, in fit self, self._fit_impl, model, train_dataloaders, val_dataloaders, datamodule, ckpt_path File "/home/haolin/anaconda3/envs/multiply/lib/python3.7/site-packages/pytorch_lightning/trainer/call.py", line 38, in _call_and_handle_interrupt return trainer_fn(*args, kwargs) File "/home/haolin/anaconda3/envs/multiply/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 650, in _fit_impl self._run(model, ckpt_path=self.ckpt_path) File "/home/haolin/anaconda3/envs/multiply/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 1103, in _run results = self._run_stage() File "/home/haolin/anaconda3/envs/multiply/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 1182, in _run_stage self._run_train() File "/home/haolin/anaconda3/envs/multiply/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 1205, in _run_train self.fit_loop.run() File "/home/haolin/anaconda3/envs/multiply/lib/python3.7/site-packages/pytorch_lightning/loops/loop.py", line 199, in run self.advance(*args, *kwargs) File "/home/haolin/anaconda3/envs/multiply/lib/python3.7/site-packages/pytorch_lightning/loops/fit_loop.py", line 267, in advance self._outputs = self.epoch_loop.run(self._data_fetcher) File "/home/haolin/anaconda3/envs/multiply/lib/python3.7/site-packages/pytorch_lightning/loops/loop.py", line 199, in run self.advance(args, kwargs) File "/home/haolin/anaconda3/envs/multiply/lib/python3.7/site-packages/pytorch_lightning/loops/epoch/training_epoch_loop.py", line 213, in advance batch_output = self.batch_loop.run(kwargs) File "/home/haolin/anaconda3/envs/multiply/lib/python3.7/site-packages/pytorch_lightning/loops/loop.py", line 199, in run self.advance(*args, kwargs) File "/home/haolin/anaconda3/envs/multiply/lib/python3.7/site-packages/pytorch_lightning/loops/batch/training_batch_loop.py", line 90, in advance outputs = self.manual_loop.run(kwargs) File "/home/haolin/anaconda3/envs/multiply/lib/python3.7/site-packages/pytorch_lightning/loops/loop.py", line 199, in run self.advance(*args, kwargs) File "/home/haolin/anaconda3/envs/multiply/lib/python3.7/site-packages/pytorch_lightning/loops/optimization/manual_loop.py", line 110, in advance training_step_output = self.trainer._call_strategy_hook("training_step", kwargs.values()) File "/home/haolin/anaconda3/envs/multiply/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 1485, in _call_strategy_hook output = fn(args, kwargs) File "/home/haolin/anaconda3/envs/multiply/lib/python3.7/site-packages/pytorch_lightning/strategies/strategy.py", line 378, in training_step return self.model.training_step(*args, kwargs) File "/data/mount_d/MultiPly/code/multiply_model.py", line 192, in training_step model_outputs = self.model(inputs) File "/home/haolin/anaconda3/envs/multiply/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl return forward_call(*input, *kwargs) File "/data/mount_d/MultiPly/code/lib/model/multiply.py", line 455, in forward weights, transmittance, alphas = render_weight_from_density(t_starts, t_ends, sigmas, ray_indices=ray_indices, n_rays=n_rays) File "/home/haolin/anaconda3/envs/multiply/lib/python3.7/site-packages/nerfacc/vol_rendering.py", line 391, in render_weight_from_density packed_info, t_starts, t_ends, sigmas File "/home/haolin/anaconda3/envs/multiply/lib/python3.7/site-packages/nerfacc/vol_rendering.py", line 633, in forward packed_info, t_starts, t_ends, sigmas File "/home/haolin/anaconda3/envs/multiply/lib/python3.7/site-packages/nerfacc/cuda/init.py", line 13, in call_cuda return getattr(_C, name)(args, **kwargs) IndexError: Dimension out of range (expected to be in range of [-1, 0], but got 1)
Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace. ''' I git clone the whole repo and didn't make change, my environment setting is: cudatoolkit: 11.1 python: 3.7 pytorch: 1.10.0+cu111 torchvision: 0.10.0+cu111 pytorch3d: 0.7.7 kaolin: 0.13.0
Thank you for the reply.