Closed foduucom closed 3 months ago
It's a problem with the Transformers library. https://github.com/huggingface/transformers/issues/30727
Would you try this?
pip install git+https://github.com/huggingface/transformers.git@d0c72c15c25e100860b6af692d084f06546a0b7a
@kadirnar Thank you for your quick response. But I am still facing same issue. š
Transformers library solved the error. Can you try deleting the Transformers library and reinstalling it? https://github.com/huggingface/transformers/pull/30748
pip install git+https://github.com/huggingface/transformers
I think the issue from transformers has been solved. But further there is something else :
2024-05-11 14:51:45,554 - INFO - We will use 90% of the memory on device 0 for storing the model, and 10% for the buffer to avoid OOM. You can set `max_memory` in to a higher value to use more memory (at your own risk).
2024-05-11 14:51:55,127 - INFO - Model loaded successfully.
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
2024-05-11 14:51:56,829 - INFO - Transcribing audio...
Traceback (most recent call last):
File "/home/ml/whisperplus/demo-test.py", line 31, in <module>
transcript = pipeline(
File "/home/ml/.local/lib/python3.10/site-packages/whisperplus/pipelines/whisper.py", line 92, in __call__
result = pipe(audio_path)
File "/home/ml/.local/lib/python3.10/site-packages/transformers/pipelines/automatic_speech_recognition.py", line 285, in __call__
return super().__call__(inputs, **kwargs)
File "/home/ml/.local/lib/python3.10/site-packages/transformers/pipelines/base.py", line 1234, in __call__
return next(
File "/home/ml/.local/lib/python3.10/site-packages/transformers/pipelines/pt_utils.py", line 124, in __next__
item = next(self.iterator)
File "/home/ml/.local/lib/python3.10/site-packages/transformers/pipelines/pt_utils.py", line 269, in __next__
processed = self.infer(next(self.iterator), **self.params)
File "/home/ml/.local/lib/python3.10/site-packages/transformers/pipelines/base.py", line 1149, in forward
model_outputs = self._forward(model_inputs, **forward_params)
File "/home/ml/.local/lib/python3.10/site-packages/transformers/pipelines/automatic_speech_recognition.py", line 506, in _forward
generate_kwargs["encoder_outputs"] = encoder(inputs, attention_mask=attention_mask)
File "/home/ml/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/ml/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "/home/ml/.local/lib/python3.10/site-packages/accelerate/hooks.py", line 166, in new_forward
output = module._old_forward(*args, **kwargs)
File "/home/ml/.local/lib/python3.10/site-packages/transformers/models/whisper/modeling_whisper.py", line 1209, in forward
layer_outputs = encoder_layer(
File "/home/ml/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/ml/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "/home/ml/.local/lib/python3.10/site-packages/accelerate/hooks.py", line 166, in new_forward
output = module._old_forward(*args, **kwargs)
File "/home/ml/.local/lib/python3.10/site-packages/transformers/models/whisper/modeling_whisper.py", line 765, in forward
hidden_states, attn_weights, _ = self.self_attn(
File "/home/ml/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/ml/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "/home/ml/.local/lib/python3.10/site-packages/transformers/models/whisper/modeling_whisper.py", line 435, in forward
query_states = self._reshape(self.q_proj(hidden_states), -1, bsz)
File "/home/ml/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/ml/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "/home/ml/.local/lib/python3.10/site-packages/accelerate/hooks.py", line 166, in new_forward
output = module._old_forward(*args, **kwargs)
File "/home/ml/.local/lib/python3.10/site-packages/hqq/core/quantize.py", line 790, in forward_aten_backprop
return HQQMatmulNoCacheDeq.apply(x, self.dequantize_aten, self.bias)
File "/home/ml/.local/lib/python3.10/site-packages/torch/autograd/function.py", line 539, in apply
return super().apply(*args, **kwargs) # type: ignore[misc]
File "/home/ml/.local/lib/python3.10/site-packages/hqq/core/quantize.py", line 257, in forward
out = torch.matmul(x, dequantize().t())
File "/home/ml/.local/lib/python3.10/site-packages/hqq/core/quantize.py", line 713, in dequantize_aten
W_est = self.dequantize_Wq_aten(W_q, meta)
File "/home/ml/.local/lib/python3.10/site-packages/hqq/core/quantize.py", line 661, in dequantize_Wq_aten
return hqq_aten.dequantize(
AttributeError: 'NoneType' object has no attribute 'dequantize'
Tested and working with RTX 3090. Did you do this?
pip uninstall transformers
pip install git+https://github.com/huggingface/transformers
i am trying it on RTX 3060. Still, i am facing the same issue. I will try it on Colab or somewhere else. Then update you!
Tomorrow I will update colab notebook and test it.
Thank you for creating this amazing package. It looks very promising. But i am facing some issue on installation on Linux Mint (Ubuntu) desktop. Where i have a NVIDIA RTX 3060 GPU with 12GB VRAM. I am just trying the same code as provided in the reference readme.
It can download the YouTube video and convert to MP3, also model is downloaded. It loaded the model into memory (not sure) but then i am stuck. Below the error. If need any trace log i can share.