search

facebookresearch / seamless_communication

Foundational Models for State-of-the-Art Speech and Text Translation
Other
10.52k stars 1.02k forks source link

RuntimeError: "LayerNormKernelImpl" not implemented for 'Half' #47

Closed aiwinapp closed 10 months ago

aiwinapp commented 10 months ago 
root@Ubuntu-2204-jammy-amd64-base ~/seamless/seamless_communication # python3 scripts/m4t/predict/predict.py привет t2tt eng --src_lang rus
INFO:__main__:Running inference on the CPU.
Using the cached checkpoint of the model 'seamlessM4T_large'. Set `force=True` to download again.
Using the cached tokenizer of the model 'seamlessM4T_large'. Set `force=True` to download again.
Using the cached checkpoint of the model 'vocoder_36langs'. Set `force=True` to download again.
Traceback (most recent call last):
  File "/root/seamless/seamless_communication/scripts/m4t/predict/predict.py", line 86, in <module>
    main()
  File "/root/seamless/seamless_communication/scripts/m4t/predict/predict.py", line 67, in main
    translated_text, wav, sr = translator.predict(
  File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/seamless_communication/models/inference/translator.py", line 209, in predict
    result = self.get_prediction(
  File "/usr/local/lib/python3.10/dist-packages/seamless_communication/models/inference/translator.py", line 120, in get_prediction
    return generator(
  File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/seamless_communication/models/unity/generator.py", line 175, in __call__
    text_output = self.t2t_generator.generate_ex(source_seqs, source_seq_lens)
  File "/usr/local/lib/python3.10/dist-packages/fairseq2/generation/text.py", line 155, in generate_ex
    return self._do_generate(source_seqs, source_seq_lens)
  File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/fairseq2/generation/text.py", line 71, in _do_generate
    encoder_output, encoder_padding_mask = self.model.encode(
  File "/usr/local/lib/python3.10/dist-packages/seamless_communication/models/unity/model.py", line 191, in encode
    return self.encoder(seqs, padding_mask)  # type: ignore[no-any-return]
  File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/fairseq2/nn/transformer/encoder.py", line 155, in forward
    seqs, padding_mask = layer(seqs, padding_mask)
  File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/fairseq2/nn/transformer/encoder_layer.py", line 175, in forward
    seqs = self._forward_self_attn(seqs, padding_mask)
  File "/usr/local/lib/python3.10/dist-packages/fairseq2/nn/transformer/encoder_layer.py", line 187, in _forward_self_attn
    seqs = self.self_attn_layer_norm(seqs)
  File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/fairseq2/nn/normalization.py", line 107, in forward
    return layer_norm(x, self.normalized_shape, self.weight, self.bias, self.eps)
  File "/usr/local/lib/python3.10/dist-packages/torch/nn/functional.py", line 2515, in layer_norm
    return torch.layer_norm(input, normalized_shape, weight, bias, eps, torch.backends.cudnn.enabled)
RuntimeError: "LayerNormKernelImpl" not implemented for 'Half'

Hello. I have a problems with running

python3 scripts/m4t/predict/predict.py привет t2tt eng --src_lang rus

Any ideas how to solve?

cbalioglu commented 10 months ago

@konstik37 Unfortunately PyTorch lacks fp16 support for many CPU-based ops. If you have a CUDA-capable GPU available, I suggest running your code on device; otherwise, you should be able to run inference on CPU in fp32.

aiwinapp commented 10 months ago

thx. I was fix it by adding converter to float32:

    self.weight = Parameter(self.weight.to(dtype=torch.float32))
    if self.bias is not None:
        self.bias = Parameter(self.bias.to(dtype=torch.float32))
cbalioglu commented 10 months ago

@kauterry @cndn fyi

Visne commented 10 months ago

@konstik37 Where did you put this code?