Re-Iterate, Fix PyTorch Benchmark and add Readme

premAI-io / benchmarks

🕹️ Performance Comparison of MLOps Engines, Frameworks, and Languages on Mainstream AI Models.

MIT License

130 stars 5 forks source link

Closed Anindyadeep closed 8 months ago

Anindyadeep commented 8 months ago

Checks:

[X] Checksetup.sh and bench.sh have Python files configured for both Mac and Linux
[X] Check if fp32 < fp16 and int8 < int4 patterns follow
[X] Check the log file name and where it is kept
[X] Check if all precisions can be covered or not and if not put an issue
[X] Check if all devices can be covered or not otherwise put an issue or find an issue
[X] Check the Benchmark README

Anindyadeep commented 8 months ago

Todo: Add a Quantized version with BitsAndBytes and check support for CPU.

Anindyadeep commented 8 months ago

Some comments:

For having a significant difference I needed to bump up the max tokens from 200 to 512
Change the prompt so that the prompt can generate something long.
Moved log files to a Logs folder. The reason is, that it makes things very cluttered.

cc: @nsosio

I am thinking to keep this pattern for the other benchmarks as well.