janhq / cortex

Drop-in, local AI alternative to the OpenAI stack. Multi-engine (llama.cpp, TensorRT-LLM, ONNX). Powers 👋 Jan
https://cortex.so
Apache License 2.0
1.78k stars 97 forks source link

Cortex Benchmark #798

Closed imtuyethan closed 2 days ago

imtuyethan commented 1 week ago

Specs

https://www.notion.so/jan-ai/Collect-User-Submitted-cortex-benchmark-data-1b10fe557e2745ff85473fd3cc227b12?pvs=4

Commands:

cortex benchmark [model_id]
louis-jan commented 5 days ago

@marknguyen1302 to update:

louis-jan commented 2 days ago

This is implemented and can be closed; please feel free to open new issues for bug tracking.