Issue on loading dspy.HFModel

[andrelima666]

Hello,

I was trying to test dspy with HuggingFace models instead of going into my Azure OpenAI. As I was trying to test this out, I'm having this issue recurrently with every model I tryout. These are the code instructions I'm passing (removed the token for HF access):

This is the error I'm facing. ( ValueError: You are trying to offload the whole model to the disk. Please use the disk_offload function instead. )

[okhat]

For local models, please use TGI or VLLM, not HFModel

[andrelima666]

So model needs to be always deployed in some sort of api/uri, it cannot be called from downloading HF model locally? 🤔 I'll try it.

[okhat]

Awesome. Let us know how it goes.