Closed YerongLi closed 3 weeks ago
I made sure the template_type is Qwen-VL template, the error persists which is to say __post_encode does not work.
===== qwfine/sft_repe.py ====
<utils.template.Qwen2VLTemplate object at 0x7f4b34eb6810>
===== qwfine/sft_repe.py ====
please use torch>=2.0
please use torch>=2.0
(qw) yerong2@ qwfine$ pip show torch
Name: torch
Version: 2.4.0
Summary: Tensors and Dynamic neural networks in Python with strong GPU acceleration
Home-page: https://pytorch.org/
Author: PyTorch Team
Author-email: packages@pytorch.org
License: BSD-3
Location: /home/yerong2/local/miniconda3/envs/qw/lib/python3.11/site-packages
Requires: filelock, fsspec, jinja2, networkx, nvidia-cublas-cu12, nvidia-cuda-cupti-cu12, nvidia-cuda-nvrtc-cu12, nvidia-cuda-runtime-cu12, nvidia-cudnn-cu12, nvidia-cufft-cu12, nvidia-curand-cu12, nvidia-cusolver-cu12, nvidia-cusparse-cu12, nvidia-nccl-cu12, nvidia-nvtx-cu12, sympy, triton, typing-extensions
Required-by: accelerate, bitsandbytes, deepspeed, evalscope, ms-opencompass, ms-vlmeval, peft, sentence-transformers, torchvision, trl, xtuner
I don't think self.post_encode is ever added in to the res_extra, https://github.com/modelscope/ms-swift/blob/7594d19188cf2fd6f592b0216ccd421179616b38/swift/llm/utils/template.py#L343
for d in data:
res_extra.append(self._post_encode(module, d))
print(' === qwfine/utils/template.py ===')
print('This branch')
print(' === qwfine/utils/template.py ===')
exit(0)
My mistake, making sure to get the correct template will get _post_encode work
Describe the bug What the bug is, and how to reproduce, better with screenshots(描述bug以及复现过程,最好有截图)
https://github.com/huggingface/transformers/blob/main/src/transformers/models/qwen2_vl/modeling_qwen2_vl.py thinks the input_embeds is None which is to say _post_encode does not work
Your hardware and system info Write your system info like CUDA version/system/GPU/torch version here(在这里给出硬件信息和系统信息,如CUDA版本,系统,GPU型号和torch版本等)
Additional context Add any other context about the problem here(在这里补充其他信息)