search

ermig1979 / Synet

A small framework to infer neural network
MIT License
140 stars 26 forks source link

Can it support ggml as BLAS layer? #34

Open znsoftm opened 1 year ago

znsoftm commented 1 year ago

ggml.ai can quantize a model to int4/8, and can seed up the inference of a model.

znsoftm commented 1 year ago

https://github.com/ggerganov/ggml