Should output total elapsed time as well.

yale-sys / prompt-cache

Modular and structured prompt caching for low-latency LLM inference

MIT License

14 stars 1 forks source link

Closed sarda-nikhil closed 8 months ago

sarda-nikhil commented 9 months ago

For completeness sake, we should also output total elapsed time.