search

triton-inference-server / client

Triton Python, C++ and Java client libraries, and GRPC-generated client examples for go, java and scala.
BSD 3-Clause "New" or "Revised" License
517 stars 224 forks source link

Reorganize metrics and data parser #712

Closed nv-hwoo closed 1 week ago

nv-hwoo commented 1 week ago

Refactor/organize the classes under llm_metics.py into separate packages. For instance, Metrics, LLMMetrics, Statistics, ProfileDataParser, and LLMProfileDataParser were all defined in llm_metrics.py. This hurts readability as the file gets bigger and bigger as we add more features. Also defining the metrics and data parsers into separate packages helps us prevent potential coupling of the two different abstractions.

This PR does/should NOT change the behavior of the code.

Before:

genai-perf/
|-- llm_metrics.py

After

genai-perf/
|-- metrics
|   |-- __init__.py
|   |-- metrics.py
|   |-- statistics.py
|   |-- llm_metrics.py
|-- profile_data_parser
|   |-- __init__.py
|   |-- profile_data_parser.py
|   |-- llm_profile_data_parser.py
dyastremsky commented 1 week ago

Great work! This will be so helpful long-term.

Looks good to me, assuming all the files stayed the same. Can approve once Elias's comments are addressed (especially regarding the removed tests).

debermudez commented 1 week ago

Gorgeous. Amazing work