neuralmagic / guidellm

Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs
Apache License 2.0
159 stars 11 forks source link

Refactor and Optimize Backend, Scheduler, and Test Suites for Improved Performance and Maintainability #27

Closed markurtz closed 2 months ago

markurtz commented 2 months ago

Summary

Introduces significant refactors and optimizations across the backend, scheduler, and test suites to enhance performance, maintainability, and code clarity. Key changes include modularization of code, simplifying complex components, and expanding unit tests to ensure robust functionality.

Details

Test Plan

Unittest Coverage Report

Name                                         Stmts   Miss  Cover   Missing
--------------------------------------------------------------------------
src/guidellm/__init__.py                         3      0   100%
src/guidellm/backend/__init__.py                 3      0   100%
src/guidellm/backend/base.py                    76      8    89%   150-155, 183, 197, 219-221
src/guidellm/backend/openai.py                  59      7    88%   103, 162-164, 178-180
src/guidellm/config.py                          55      0   100%
src/guidellm/core/__init__.py                    6      0   100%
src/guidellm/core/distribution.py               92      0   100%
src/guidellm/core/report.py                      7      0   100%
src/guidellm/core/request.py                    10      0   100%
src/guidellm/core/result.py                    125      2    98%   104, 259
src/guidellm/core/serializable.py               64      1    98%   105
src/guidellm/executor/__init__.py                3      0   100%
src/guidellm/executor/base.py                   55      0   100%
src/guidellm/executor/profile_generator.py     113     16    86%   98, 149-151, 173-175, 207-209, 287-289, 295-297
src/guidellm/logger.py                          17      0   100%
src/guidellm/main.py                            55     55     0%   1-172
src/guidellm/request/__init__.py                 5      0   100%
src/guidellm/request/base.py                    67      0   100%
src/guidellm/request/emulated.py               130      3    98%   64, 79, 90
src/guidellm/request/file.py                    26      0   100%
src/guidellm/request/transformers.py            31      0   100%
src/guidellm/scheduler/__init__.py               3      0   100%
src/guidellm/scheduler/base.py                 138     24    83%   294-328, 334, 352-355
src/guidellm/scheduler/load_generator.py        74      5    93%   104-105, 139, 165, 175
src/guidellm/utils/__init__.py                   4      0   100%
src/guidellm/utils/injector.py                  20      0   100%
src/guidellm/utils/text.py                     201     22    89%   79-81, 86-88, 155-156, 166, 186, 190-191, 212-213, 245, 249, 304-305, 324, 344, 385, 443
src/guidellm/utils/transformers.py              53      0   100%
--------------------------------------------------------------------------
TOTAL                                         1495    143    90%