Closed andrelima666 closed 1 month ago
For local models, please use TGI or VLLM, not HFModel
So model needs to be always deployed in some sort of api/uri, it cannot be called from downloading HF model locally? 🤔 I'll try it.
Awesome. Let us know how it goes.
Hello,
I was trying to test dspy with HuggingFace models instead of going into my Azure OpenAI. As I was trying to test this out, I'm having this issue recurrently with every model I tryout. These are the code instructions I'm passing (removed the token for HF access):
This is the error I'm facing. ( ValueError: You are trying to offload the whole model to the disk. Please use the
disk_offload
function instead. )