precision error

[dillfrescott]

(seamless) root@55d07513038c:~/seamless_communication# m4t_predict mirror.wav s2tt eng --model_name seamlessM4T_large
2023-08-23 04:07:17,769 INFO -- m4t_scripts.predict.predict: Running inference on the CPU.
Using the cached checkpoint of the model 'seamlessM4T_large'. Set `force=True` to download again.
Using the cached tokenizer of the model 'seamlessM4T_large'. Set `force=True` to download again.
Using the cached checkpoint of the model 'vocoder_36langs'. Set `force=True` to download again.
Traceback (most recent call last):
  File "/root/miniconda3/envs/seamless/bin/m4t_predict", line 8, in <module>
    sys.exit(main())
  File "/root/miniconda3/envs/seamless/lib/python3.10/site-packages/m4t_scripts/predict/predict.py", line 70, in main
    translated_text, wav, sr = translator.predict(
  File "/root/miniconda3/envs/seamless/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/root/miniconda3/envs/seamless/lib/python3.10/site-packages/seamless_communication/models/inference/translator.py", line 209, in predict
    result = self.get_prediction(
  File "/root/miniconda3/envs/seamless/lib/python3.10/site-packages/seamless_communication/models/inference/translator.py", line 120, in get_prediction
    return generator(
  File "/root/miniconda3/envs/seamless/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/root/miniconda3/envs/seamless/lib/python3.10/site-packages/seamless_communication/models/unity/generator.py", line 173, in __call__
    text_output = self.s2t_generator.generate_ex(source_seqs, source_seq_lens)
  File "/root/miniconda3/envs/seamless/lib/python3.10/site-packages/fairseq2/generation/text.py", line 155, in generate_ex
    return self._do_generate(source_seqs, source_seq_lens)
  File "/root/miniconda3/envs/seamless/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/root/miniconda3/envs/seamless/lib/python3.10/site-packages/fairseq2/generation/text.py", line 71, in _do_generate
    encoder_output, encoder_padding_mask = self.model.encode(
  File "/root/miniconda3/envs/seamless/lib/python3.10/site-packages/seamless_communication/models/unity/model.py", line 190, in encode
    seqs, padding_mask = self.encoder_frontend(seqs, seq_lens)
  File "/root/miniconda3/envs/seamless/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/root/miniconda3/envs/seamless/lib/python3.10/site-packages/fairseq2/models/wav2vec2/frontend.py", line 130, in forward
    seqs, seq_lens = self.extract_features(seqs, seq_lens)
  File "/root/miniconda3/envs/seamless/lib/python3.10/site-packages/fairseq2/models/wav2vec2/frontend.py", line 163, in extract_features
    seqs = self.post_extract_layer_norm(seqs)
  File "/root/miniconda3/envs/seamless/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/root/miniconda3/envs/seamless/lib/python3.10/site-packages/fairseq2/nn/normalization.py", line 107, in forward
    return layer_norm(x, self.normalized_shape, self.weight, self.bias, self.eps)
  File "/root/miniconda3/envs/seamless/lib/python3.10/site-packages/torch/nn/functional.py", line 2515, in layer_norm
    return torch.layer_norm(input, normalized_shape, weight, bias, eps, torch.backends.cudnn.enabled)
RuntimeError: "LayerNormKernelImpl" not implemented for 'Half'

Any ideas?

[cbalioglu]

@dillfrescott from the log output, looks like you are running the inference code on CPU. Unfortunately PyTorch lacks fp16 support for many CPU-based ops. I suggest running inference on a CUDA device if you want to run in fp16, or use fp32 if you need CPU inference.

[dillfrescott]

Oh okay. Thank you!

[cbalioglu]

You are welcome!