Upload Open-Chat-Kit models at int4 on huggingface and add the URL mapping to interface.py

NolanoOrg / cformers

SoTA Transformers with C-backend for fast inference on your CPU.

MIT License

311 stars 29 forks source link

Closed Ayushk4 closed 1 year ago

Ayushk4 commented 1 year ago

Since GPT-NeoX support (Pythia based Open Assistant models) has been confirmed by #24 , we can port other chat-models.

Instructions:

Port to GGML Format: python3 cformers/cpp/converters/convert_gptneox_to_ggml.py <MODEL_CARD> <SAVE_PATH> 1
Quantize cd cformers/cpp && make , ./quantize SAVE_PATH QUANTIZE_SAVE_PATH 2
Upload to huggingface, update cformers/interface.py with path and update README

HCBlackFox commented 1 year ago

I will help with that today.

HCBlackFox commented 1 year ago

I`m on the finall stage, uploading to the HF.

HCBlackFox commented 1 year ago

tejasvaidhyadev commented 1 year ago

Thanks.

Ayushk4 commented 1 year ago

Thanks, a lot!