It seems that the move_to_gpu & move_to_cpu is not working as expected in the branch fast_inference.

SetoKaiba commented 1 month ago

https://github.com/RVC-Boss/GPT-SoVITS/blob/fast_inference_/api_v3.py#L327-L343

It will always move to cpu. The custom settings device is cuda. The precision is half.

When the move_to_cpu is called, the custom settings device became cpu and the precision kept half. But according to lines below, half is not supported for cpu.

https://github.com/RVC-Boss/GPT-SoVITS/blob/fast_inference_/GPT_SoVITS/TTS_infer_pack/TTS.py#L311-L313

I try to add the lines below to move_to_cpu. https://github.com/RVC-Boss/GPT-SoVITS/blob/fast_inference_/api_v3.py#L348-L351

    tts.enable_half_precision(False)

I try to add the lines below to move_to_gpu. https://github.com/RVC-Boss/GPT-SoVITS/blob/fast_inference_/api_v3.py#L354-L356

    tts.enable_half_precision(tts_config.is_half)

But then the first tts request is working. The second isn't. Here's the error of the second request.

{
    "message": "tts failed",
    "Exception": "Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument index in method wrapper_CUDA__index_select)"
}

@KevinZhang19870314