search

hegelai / prompttools

Open-source tools for prompt testing and experimentation, with support for both LLMs (e.g. OpenAI, LLaMA) and vector databases (e.g. Chroma, Weaviate, LanceDB).
http://prompttools.readthedocs.io
Apache License 2.0
2.55k stars 216 forks source link

Issue with HuggingFaceHubExperiment: Unable to Use Models Other Than gpt2 #110

Open shrijayan opened 6 months ago

shrijayan commented 6 months ago

⁉️ Discussion/Question

I'm currently facing an issue while attempting to use the HuggingFaceHubExperiment for testing models from Hugging Face. Specifically, it seems that only the gpt2 model is functional in this experiment, while other models from Hugging Face do not work. I am trying to use other models available from HuggingFace.

I've tried inputting Model IDs from Hugging Face, and I've also explored the available models on https://huggingface.co/api/models. Unfortunately, none of the models, aside from gpt2, seem to be compatible with the HuggingFaceHubExperiment.

Is there a specific reason for this limitation, or am I overlooking something in the setup process? I would greatly appreciate any guidance or insights you could provide to help resolve this issue.

steventkrawczyk commented 6 months ago

Thanks for bringing this issue up, I'm looking into it

shrijayan commented 6 months ago

Thanks for bringing this issue up, I'm looking into it

Thanks for looking up!

shrijayan commented 6 months ago

Thanks for bringing this issue up, I'm looking into it

Are there other possibilities to use all other huggingface models?

shrijayan commented 6 months ago

@steventkrawczyk I think its all because of the endpoint where it looks!

steventkrawczyk commented 6 months ago

Which model in particular are you trying to use? The huggingface hub experiment uses the Inference API, so it will only support models which that API supports. Is there a model you can call through the inference API, but are running into problems experimenting with prompttools? Can you post your error here?

shrijayan commented 6 months ago

Which model in particular are you trying to use? The huggingface hub experiment uses the Inference API, so it will only support models which that API supports. Is there a model you can call through the inference API, but are running into problems experimenting with prompttools? Can you post your error here?

I have also searched on https://huggingface.co/api/models, and it seems that the model present there, including GPT-2, does not support my requirements. I am certain that GPT-2 fetches the model from this link. However, any other model from the same link, such as GPT-2 Large, is also not working.