how to use a finetuned nonlocal model?

[nullonesix]

the build script and build.py expect a .pt file

my checkpoint looks like:

root@28f5bb431151:/app# ls checkpoint-4000/ config.json model-00001-of-00002.safetensors model.safetensors.index.json preprocessor_config.json scheduler.pt training_args.bin generation_config.json model-00002-of-00002.safetensors optimizer.pt rng_state.pth trainer_state.json

what must i do to properly convert this checkpoint to be compatible with the build script which seems to in no way accommodate this?

[makaveli10]

@nullonesix I suppose you want to use the TensorRT-LLM backend? To do that you would need to convert your safetensors checkpoint to pytorch checkpoint (.pt), this might be of some help in converting the checkpint => https://github.com/pytorch-labs/gpt-fast/blob/main/scripts/convert_hf_checkpoint.py

And then you would have to follow TensorRT-LLM documentation to convert you custome model to tensorrt. Finally, you should be able to run whisper-live with your custom tensorrt model.