search

pytorch-labs / gpt-fast

Simple and efficient pytorch-native transformer text generation in <1000 LOC of python.
BSD 3-Clause "New" or "Revised" License
5.56k stars 505 forks source link

Throughput Benchmark Scripts #173

Closed HanGuo97 closed 4 months ago

HanGuo97 commented 4 months ago

Hi, thanks for this amazing library!

Just curious, where can I find the script to produce the benchmarking numbers (e.g., tokens/s, bandwidth) shown in the README?

Chillee commented 4 months ago

The instructions provided to run generate.py should repro the benchmarking numbers in the README.

HanGuo97 commented 4 months ago

Ah, I just realized I missed this, thanks for the clarifications!