search

triton-inference-server / client

Triton Python, C++ and Java client libraries, and GRPC-generated client examples for go, java and scala.
BSD 3-Clause "New" or "Revised" License
527 stars 225 forks source link

update error reporting #597

Closed debermudez closed 3 months ago

debermudez commented 3 months ago

Attempt to improve the logging for genai-perf.

Some example output Using a modified profile_export.json that removed response timestamps from the first request entry. Before: genai-perf - ERROR - 'response_timestamps'

After:

Traceback (most recent call last):
  File "/workspace/client/src/c++/perf_analyzer/genai-perf/genai_perf/main.py", line 147, in run
    data_parser = calculate_metrics(args, tokenizer)
  File "/workspace/client/src/c++/perf_analyzer/genai-perf/genai_perf/main.py", line 89, in calculate_metrics
    return LLMProfileDataParser(
  File "/workspace/client/src/c++/perf_analyzer/genai-perf/genai_perf/llm_metrics.py", line 475, in __init__
    super().__init__(filename)
  File "/workspace/client/src/c++/perf_analyzer/genai-perf/genai_perf/llm_metrics.py", line 406, in __init__
    self._parse_profile_data(data)
  File "/workspace/client/src/c++/perf_analyzer/genai-perf/genai_perf/llm_metrics.py", line 416, in _parse_profile_data
    metrics = self._parse_requests(requests)
  File "/workspace/client/src/c++/perf_analyzer/genai-perf/genai_perf/llm_metrics.py", line 489, in _parse_requests
    res_timestamps = request["response_timestamps"]
KeyError: 'response_timestamps'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/workspace/client/src/c++/perf_analyzer/genai-perf/genai_perf/main.py", line 157, in main
    run()
  File "/workspace/client/src/c++/perf_analyzer/genai-perf/genai_perf/main.py", line 151, in run
    raise GenAIPerfException(e)
genai_perf.exceptions.GenAIPerfException: 'response_timestamps'
2024-04-19 16:35 [ERROR] genai_perf.main:161 - 'response_timestamps'

The error message now contains time info, module where the error occurred, and line numbers. The console will also display the traceback for easier debugging.

rmccorm4 commented 3 months ago

Couple requests to make it easier to grok the cosmetic and functional changes:

  1. Can you add before and after outputs of stdout logging for the happy path to the description?

  2. Can you also show an example of output from an error? Does it only go to file, stdout, or both?

debermudez commented 3 months ago

(edited by Ryan): Moved to thread: https://github.com/triton-inference-server/client/pull/597#discussion_r1572693154