yestinl
commented
1 month ago @godguy23 This issue appears to be the same as issue https://github.com/Tencent/HunyuanDiT/issues/37. We will close this issue for now. If there are further questions, feel free to reopen it.
运行python sample_t2i.py --prompt "渔舟唱晚" 报错，下面是错误信息，请问怎么解决呀，网上没找到解决方案，谢谢。下面是错误信息。 flash_attn import failed: No module named 'flash_attn' 2024-05-17 09:22:52.036 | INFO | hydit.inference:init:160 - Got text-to-image model root path: ckpts/t2i 2024-05-17 09:22:52.061 | INFO | hydit.inference:init:172 - Loading CLIP Text Encoder... 2024-05-17 09:22:56.856 | INFO | hydit.inference:init:175 - Loading CLIP Text Encoder finished 2024-05-17 09:22:56.856 | INFO | hydit.inference:init:178 - Loading CLIP Tokenizer... 2024-05-17 09:22:56.890 | INFO | hydit.inference:init:181 - Loading CLIP Tokenizer finished 2024-05-17 09:22:56.891 | INFO | hydit.inference:init:184 - Loading T5 Text Encoder and T5 Tokenizer... You are using the default legacy behaviour of the <class 'transformers.models.t5.tokenization_t5.T5Tokenizer'>. This is expected, and simply means that the:34 - Prompt Enhancement...
Traceback (most recent call last):
File "sample_t2i.py", line 35, in
success, enhanced_prompt = enhancer(args.prompt)
File "/home/hxt/LLM/Tencent-HunyuanDit/HunyuanDiT/dialoggen/dialoggen_demo.py", line 145, in call
enhanced_prompt = eval_model(
File "/home/hxt/LLM/Tencent-HunyuanDit/HunyuanDiT/dialoggen/dialoggen_demo.py", line 113, in eval_model
output_ids = models["model"].generate(
File "/home/hxt/miniconda3/envs/HunyuanDiT/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
return func(*args, kwargs)
File "/home/hxt/LLM/Tencent-HunyuanDit/HunyuanDiT/dialoggen/llava/model/language_model/llava_mistral.py", line 125, in generate
) = self.prepare_inputs_labels_for_multimodal(
File "/home/hxt/LLM/Tencent-HunyuanDit/HunyuanDiT/dialoggen/llava/model/llava_arch.py", line 157, in prepare_inputs_labels_for_multimodal
image_features = self.encode_images(concat_images)
File "/home/hxt/LLM/Tencent-HunyuanDit/HunyuanDiT/dialoggen/llava/model/llava_arch.py", line 141, in encode_images
image_features = self.get_model().get_vision_tower()(images)
File "/home/hxt/miniconda3/envs/HunyuanDiT/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
return forward_call(*input, *kwargs)
File "/home/hxt/miniconda3/envs/HunyuanDiT/lib/python3.8/site-packages/accelerate/hooks.py", line 166, in new_forward
output = module._old_forward(args, kwargs)
File "/home/hxt/miniconda3/envs/HunyuanDiT/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
return func(*args, kwargs)
File "/home/hxt/LLM/Tencent-HunyuanDit/HunyuanDiT/dialoggen/llava/model/multimodal_encoder/clip_encoder.py", line 54, in forward
image_forward_outs = self.vision_tower(images.to(device=self.device, dtype=self.dtype), output_hidden_states=True)
File "/home/hxt/miniconda3/envs/HunyuanDiT/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
return forward_call(*input, *kwargs)
File "/home/hxt/miniconda3/envs/HunyuanDiT/lib/python3.8/site-packages/accelerate/hooks.py", line 161, in new_forward
args, kwargs = module._hf_hook.pre_forward(module, args, kwargs)
File "/home/hxt/miniconda3/envs/HunyuanDiT/lib/python3.8/site-packages/accelerate/hooks.py", line 356, in pre_forward
return send_to_device(args, self.execution_device), send_to_device(
File "/home/hxt/miniconda3/envs/HunyuanDiT/lib/python3.8/site-packages/accelerate/utils/operations.py", line 180, in send_to_device
return honor_type(
File "/home/hxt/miniconda3/envs/HunyuanDiT/lib/python3.8/site-packages/accelerate/utils/operations.py", line 81, in honor_type
return type(obj)(generator)
File "/home/hxt/miniconda3/envs/HunyuanDiT/lib/python3.8/site-packages/accelerate/utils/operations.py", line 181, in
tensor, (send_to_device(t, device, non_blocking=non_blocking, skip_keys=skip_keys) for t in tensor)
File "/home/hxt/miniconda3/envs/HunyuanDiT/lib/python3.8/site-packages/accelerate/utils/operations.py", line 174, in send_to_device
raise error
File "/home/hxt/miniconda3/envs/HunyuanDiT/lib/python3.8/site-packages/accelerate/utils/operations.py", line 158, in send_to_device
return tensor.to(device, non_blocking=non_blocking)
NotImplementedError: Cannot copy out of meta tensor; no data!
legacy(previous) behavior will be used so nothing changes for you. If you want to use the new behaviour, setlegacy=False. This should only be set if you understand what it means, and thoroughly read the reason why this was added as explained in https://github.com/huggingface/transformers/pull/24565 /home/hxt/miniconda3/envs/HunyuanDiT/lib/python3.8/site-packages/transformers/convert_slow_tokenizer.py:515: UserWarning: The sentencepiece tokenizer that you are converting to a fast tokenizer uses the byte fallback option which is not implemented in the fast tokenizers. In practice this means that the fast version of the tokenizer can produce unknown tokens whereas the sentencepiece version would have converted these unknown tokens into a sequence of byte tokens matching the original piece of text. warnings.warn( You are using a model of type mt5 to instantiate a model of type t5. This is not supported for all configurations of models and can yield errors. 2024-05-17 09:23:21.105 | INFO | hydit.inference:init:188 - Loading t5_text_encoder and t5_tokenizer finished 2024-05-17 09:23:21.106 | INFO | hydit.inference:init:191 - Loading VAE... 2024-05-17 09:23:21.462 | INFO | hydit.inference:init:194 - Loading VAE finished 2024-05-17 09:23:21.462 | INFO | hydit.inference:init:198 - Building HunYuan-DiT model... 2024-05-17 09:23:21.828 | INFO | hydit.modules.models:init:229 - Number of tokens: 4096 2024-05-17 09:23:34.236 | INFO | hydit.inference:init:218 - Loading model checkpoint ckpts/t2i/model/pytorch_model_ema.pt... 2024-05-17 09:23:38.890 | INFO | hydit.inference:init:229 - Loading inference pipeline... 2024-05-17 09:23:38.927 | INFO | hydit.inference:init:231 - Loading pipeline finished 2024-05-17 09:23:38.927 | INFO | hydit.inference:init:235 - ================================================== 2024-05-17 09:23:38.927 | INFO | hydit.inference:init:236 - Model is ready.2024-05-17 09:23:38.927 | INFO | hydit.inference:init:237 - ================================================== 2024-05-17 09:23:39.014 | INFO | main:inferencer:21 - Loading DialogGen model (for prompt enhancement)... /home/hxt/miniconda3/envs/HunyuanDiT/lib/python3.8/site-packages/huggingface_hub/file_download.py:1132: FutureWarning:
resume_downloadis deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, useforce_download=True. warnings.warn( /home/hxt/miniconda3/envs/HunyuanDiT/lib/python3.8/site-packages/accelerate/utils/modeling.py:1363: UserWarning: Current model requires 318774152 bytes of buffer for offloaded layers, which seems does not fit any GPU's remaining memory. If you are experiencing a OOM later, please consider using offload_buffers=True. warnings.warn( Loading checkpoint shards: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:04<00:00, 1.23s/it] WARNING:root:Some parameters are on the meta device device because they were offloaded to the cpu. 2024-05-17 09:24:16.579 | INFO | main:inferencer:23 - DialogGen model loaded. 2024-05-17 09:24:16.579 | INFO | main: