search

spcl / graph-of-thoughts

Official Implementation of "Graph of Thoughts: Solving Elaborate Problems with Large Language Models"
https://arxiv.org/pdf/2308.09687.pdf
Other
2.19k stars 159 forks source link

Can I load other models on huggingface to use this nice tool please? #22

Closed WCSY-YG closed 11 months ago

WCSY-YG commented 11 months ago

i'd like to use qwen model to perform GoT reasoning，thank you

nblach commented 11 months ago

Hi @WCSY-YG,

In theory, you can use any model with GoT. We show how you can use models hosted on HuggingFace (HF) at the example of Llama2: https://github.com/spcl/graph-of-thoughts/blob/f508aef4fdfa94adc505f5a7745e6088bdf7889c/graph_of_thoughts/language_models/llamachat_hf.py If you want to use a different HF model, you should only need to change the hf_model_id in the configurations as it uses AutoTokenizer and AutoModel. However, we currently always quantise, which you would need to update if this is not supported/wanted. Hope that helps!