search

mlc-ai / llm-perf-bench

Apache License 2.0
109 stars 12 forks source link

Enable Docker for Llama.cpp Benchmarking #5

Closed zxybazh closed 1 year ago

zxybazh commented 1 year ago

Add docker scripts to benchmark llama.cpp performance for llm models on cuda.

zxybazh commented 1 year ago

Ready for review! CC: @junrushao

junrushao commented 1 year ago

Thanks @zxybazh! Will review tomorrow by trying to actually run it on my 3090ti :)