jdf-prog
commented
3 hours ago It seems to be that my custom qwen2vl are not applied to to the liger kernel.
def apply_liger_kernel_to_qwen2_vl(
cross_entropy: bool = False,
fused_linear_cross_entropy: bool = True,
rms_norm: bool = True,
layer_norm: bool = True,
swiglu: bool = True,
model: PreTrainedModel = None,
) -> None:
"""
Apply Liger kernels to replace original implementation in HuggingFace Qwen2-VL models.
NOTE: Qwen2-VL is not available in transformers<4.45.0
Args:
cross_entropy (bool): Whether to apply Liger's cross entropy loss. Default is False.
fused_linear_cross_entropy (bool):
Whether to apply Liger's fused linear cross entropy loss. Default is True.
`cross_entropy` and `fused_linear_cross_entropy` cannot both be True.
If `fused_linear_cross_entropy` is True, the logits will not be materialized but more memory efficient.
rms_norm (bool): Whether to apply Liger's RMSNorm. Default is True.
layer_norm (bool): Whether to apply Liger's LayerNorm. Default is True.
swiglu (bool): Whether to apply Liger's SwiGLU MLP. Default is True.
model (PreTrainedModel): The model instance to apply Liger kernels to, if the model has already been
loaded. Default is None.
"""
assert not (
cross_entropy and fused_linear_cross_entropy
), "cross_entropy and fused_linear_cross_entropy cannot both be True."
# from transformers.models.qwen2_vl import modeling_qwen2_vl
# from transformers.models.qwen2_vl.modeling_qwen2_vl import Qwen2VLModel
from my_folder.models.qwen2_vl import modeling_qwen2_vl
from my_folder.models.qwen2_vl.modeling_qwen2_vl import Qwen2VLModelAfter I changed to this the problem is solved.
ByronHsu
🐛 Describe the bug
It seems the applying liger to qwen2vl will cause this bug.
Reproduce
No response
Versions
Environment Report:
Operating System: Linux-5.15.0-60-generic-x86_64-with-glibc2.31 Python version: 3.10.15 PyTorch version: 2.5.1+cu124 CUDA version: 12.4 Triton version: 3.1.0 Transformers version: 4.46.2