Open hesz94 opened 1 year ago
I didn't even notice that flag, wow.
in diffusion_decoder.py, use_fp16
is only used to assign DiffusionTts().enable_fp16
, which in turn is only used here:
https://github.com/152334H/tortoise-tts-fast/blob/main/tortoise/models/diffusion_decoder.py#L303-306
I think the previous time I tried to wrap an autocast around the whole model, I got an error somewhere I couldn't fix. I'll pass forward the fp16 flag for the time being
Yeah I tried manually converting the entire model and failed too. It should be doable, just not going to be as easy as we'd wish
Diffuser can supposedly be switched to half precision when it's being built (line 239 in api.py) tried this on a 3090ti and got marginal (5% ish iirc) speed improvement - presumably this switch isn't properly implemented and it switches back to full precision. Will have to do a proper trace of the diffuser run to see exactly what's going on.