3. Run CygWin Terminal and go to the cformers/cpp folder
4. Run make comand
5. Change line 127 in interface.py to
command = ["./cpp/main.exe", self.cpp_model_name,
That is all.
Example of script to run OA model:
from interface import AutoInference as AI
ai = AI('OpenAssistant/oasst-sft-1-pythia-12b')
x = ai.generate("<|prompter|>What's the Earth total population<|endoftext|><|assistant|>", num_tokens_to_generate=100)
print(x['token_str'])
Also, I atached zip with compiled files and changeds in interface,py
cformers.zip
1. Download Cygwin and Strawberry Perl (Or any other gcc I think, but then change line 16 in make file to your compiler)
https://www.cygwin.com/ https://strawberryperl.com/
and
2. Add lines in make file:
Also delete line 200.
3. Run CygWin Terminal and go to the cformers/cpp folder
4. Run make comand
5. Change line 127 in interface.py to
That is all.
Example of script to run OA model:
Also, I atached zip with compiled files and changeds in interface,py cformers.zip