search

NVIDIA / OpenSeq2Seq

Toolkit for efficient experimentation with Speech Recognition, Text2Speech and NLP
https://nvidia.github.io/OpenSeq2Seq
Apache License 2.0
1.54k stars 369 forks source link

Mixed precision model cannot be run in infer mode #394

Closed oscarpang closed 5 years ago

oscarpang commented 5 years ago

Hi,

After I trained Tacotron with mixed precesion, I cannot run infer mode on it. It prompt a TypeError.

File "/home/oscar/.local/lib/python3.5/site-packages/tensorflow/python/framework/op_def_library.py", line 609, in _apply_op_helper param_name=input_name) File "/home/oscar/.local/lib/python3.5/site-packages/tensorflow/python/framework/op_def_library.py", line 60, in _SatisfiesTypeConstraint ", ".join(dtypes.as_dtype(x).name for x in allowed_list))) TypeError: Value passed to parameter 'x' has DataType float16 not in list of allowed values: float32

Thanks.

okuchaiev commented 5 years ago

which GPU are you using? Also, did you train using float32?

oscarpang commented 5 years ago

I am using 1080 Ti.

I am training it with dtype = "mixed"

okuchaiev commented 5 years ago

Sorry, mixed precision training needs Tensor Cores which are only present on Volta (V100, Titan V, etc.) or Turing (2080, 2080Ti etc.) based GPUs.

oscarpang commented 5 years ago

I understand that the training would need Volta or Turing. Does the inference also need that?

blisc commented 5 years ago

You cannot currently mix training and eval dtype. Eg, if you trained in float32, you must run eval and inference in float32.

Yes, you require Volta or Turing for running eval/infer with "mixed" models. Otherwise, iirc, the float16 operations gets offloaded to the CPU.

oscarpang commented 5 years ago

Thanks.