thuliu-yt16
commented
1 year ago Should fixed in #258. We also recommend you use transformers==4.28.1 to avoid potential errors when using DeepFloyd.
Closed adsf0427 closed 1 year ago
thuliu-yt16
commented
1 year ago Should fixed in #258. We also recommend you use transformers==4.28.1 to avoid potential errors when using DeepFloyd.
encountered a bug when running on colab： 2023-07-27 13:02:02.977976: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT /usr/local/lib/python3.10/dist-packages/controlnet_aux/mediapipe_face/mediapipe_face_common.py:7: UserWarning: The module 'mediapipe' is not installed. The package will have limited functionality. Please install it using the command: pip install 'mediapipe' warnings.warn( Global seed set to 0 [INFO] ModelCheckpoint(save_last=True, save_top_k=-1, monitor=None) will duplicate the last checkpoint saved. [INFO] Using 16bit Automatic Mixed Precision (AMP) [INFO] GPU available: True (cuda), used: True [INFO] TPU available: False, using: 0 TPU cores [INFO] IPU available: False, using: 0 IPUs [INFO] HPU available: False, using: 0 HPUs [INFO] LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0] [INFO] | Name | Type | Params
0 | geometry | ImplicitVolume | 12.6 M 1 | material | DiffuseWithPointLightMaterial | 0
2 | background | NeuralEnvironmentMapBackground | 448
3 | renderer | NeRFVolumeRenderer | 0
12.6 M Trainable params 0 Non-trainable params 12.6 M Total params 50.419 Total estimated model params size (MB) [INFO] Validation results will be saved to outputs/dreamfusion-sd/a_zoomed_out_DSLR_photo_of_a_baby_bunny_sitting_on_top_of_a_stack_of_pancakes@20230727-130208/save [INFO] Using prompt [a zoomed out DSLR photo of a baby bunny sitting on top of a stack of pancakes] and negative prompt [] [INFO] Using view-dependent prompts [side]:[a zoomed out DSLR photo of a baby bunny sitting on top of a stack of pancakes, side view] [front]:[a zoomed out DSLR photo of a baby bunny sitting on top of a stack of pancakes, front view] [back]:[a zoomed out DSLR photo of a baby bunny sitting on top of a stack of pancakes, back view] [overhead]:[a zoomed out DSLR photo of a baby bunny sitting on top of a stack of pancakes, overhead view] Traceback (most recent call last): File "/content/threestudio/launch.py", line 237, in
main(args, extras)
File "/content/threestudio/launch.py", line 180, in main
trainer.fit(system, datamodule=dm, ckpt_path=cfg.resume)
File "/usr/local/lib/python3.10/dist-packages/pytorch_lightning/trainer/trainer.py", line 529, in fit
call._call_and_handle_interrupt(
File "/usr/local/lib/python3.10/dist-packages/pytorch_lightning/trainer/call.py", line 42, in _call_and_handle_interrupt
return trainer_fn(*args, kwargs)
File "/usr/local/lib/python3.10/dist-packages/pytorch_lightning/trainer/trainer.py", line 568, in _fit_impl
self._run(model, ckpt_path=ckpt_path)
File "/usr/local/lib/python3.10/dist-packages/pytorch_lightning/trainer/trainer.py", line 954, in _run
call._call_lightning_module_hook(self, "on_fit_start")
File "/usr/local/lib/python3.10/dist-packages/pytorch_lightning/trainer/call.py", line 144, in _call_lightning_module_hook
output = fn(*args, *kwargs)
File "/content/threestudio/threestudio/systems/dreamfusion.py", line 32, in on_fit_start
self.prompt_processor = threestudio.find(self.cfg.prompt_processor_type)(
File "/content/threestudio/threestudio/utils/base.py", line 63, in init
self.configure(args, kwargs)
File "/content/threestudio/threestudio/models/prompt_processors/base.py", line 335, in configure
self.prepare_text_embeddings()
File "/usr/local/lib/python3.10/dist-packages/lightning_utilities/core/rank_zero.py", line 32, in wrapped_fn
return fn(*args, kwargs)
File "/content/threestudio/threestudio/models/prompt_processors/base.py", line 382, in prepare_text_embeddings
self.spawn_func(
File "/content/threestudio/threestudio/models/prompt_processors/stable_diffusion_prompt_processor.py", line 91, in spawn_func
text_embeddings = text_encoder(tokens.input_ids)[0]
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, *kwargs)
File "/usr/local/lib/python3.10/dist-packages/transformers/models/clip/modeling_clip.py", line 823, in forward
return self.text_model(
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(args, kwargs)
File "/usr/local/lib/python3.10/dist-packages/transformers/models/clip/modeling_clip.py", line 731, in forward
hidden_states = self.embeddings(input_ids=input_ids, position_ids=position_ids)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, *kwargs)
File "/usr/local/lib/python3.10/dist-packages/transformers/models/clip/modeling_clip.py", line 229, in forward
inputs_embeds = self.token_embedding(input_ids)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/sparse.py", line 162, in forward
return F.embedding(
File "/usr/local/lib/python3.10/dist-packages/torch/nn/functional.py", line 2210, in embedding
return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu! (when checking argument for argument index in method wrapper_CUDA__index_select)