Can I load modules and checkpoints from local files？

yuchenlin / LLM-Blender

[ACL2023] We introduce LLM-Blender, an innovative ensembling framework to attain consistently superior performance by leveraging the diverse strengths of multiple open-source LLMs. LLM-Blender cut the weaknesses through ranking and integrate the strengths through fusing generation to enhance the capability of LLMs.

https://yuchenlin.xyz/LLM-Blender/

Apache License 2.0

882 stars 76 forks source link

Can I load modules and checkpoints from local files？ #27

Closed wangq326 closed 2 weeks ago

wangq326 commented 1 month ago

Hello, your work is exceptional, and I am very interested in it. However, I am unable to load models from Hugging Face. Is it possible for me to download the checkpoints you have released and load them from local files?

Looking forward to your reply. Best regards!

jdf-prog commented 1 month ago

Can you post what errors you have encountered during the loading?

jdf-prog commented 1 month ago

import os
os.environ["CUDA_VISIBLE_DEVICES"] = "0"
import llm_blender
blender = llm_blender.Blender()
# Load Ranker
blender.loadranker("llm-blender/PairRM") # load ranker checkpoint

For me, the above code, and the codes in the readme works fine for me. It will be good if you can provide more error information so I can help you solve it.

jdf-prog commented 1 month ago

About the local files, of course it's possible. simply replace the model_name to your local checkpoint path.