Closed subby2006 closed 10 months ago
as of right now you can only load gpt-j,opt models on tpu colab anything such as llama or mistral isnt possible
This is incorrect for Llama which can be loaded on colab on the United version. You load compatible custon models by entering their HF 16-but name into the model field. But we can not provide support for models in the safetensors format so your model needs to be in a pytorch_model.bin format.
In general the TPU is a legacy feature for us and we only try to keep it functional until colab inevitably bans our UI like they have done others. For the best experience use the GpU colab instead.
Is it possible to edit the notebook and load custom models onto ColabKobold TPU. If so, what formats must the model be in. There are a few models listed on the readme but aren’t available through the notebook so was wondering