search

ggerganov / ggml

Tensor library for machine learning
MIT License
10.96k stars 1.01k forks source link

MosaicML MPT-7B #136

Open ehartford opened 1 year ago

ehartford commented 1 year ago

As was suggested in https://github.com/ggerganov/llama.cpp/issues/1333

Create a basic inference example for MosaicML MPT-7B model

lukasmoellerch commented 1 year ago

I'll see how much my replit branch (from #131) would have to be adjusted... I think the main difference is the tokenizer and qkv clamping.

lukasmoellerch commented 1 year ago

Looks decent, I'll create one PR for both then.

Screenshot 2023-05-10 at 20 11 37