search

triton-inference-server / client

Triton Python, C++ and Java client libraries, and GRPC-generated client examples for go, java and scala.
BSD 3-Clause "New" or "Revised" License
527 stars 225 forks source link

Add more metadata to profile export JSON file #627

Closed nv-hwoo closed 2 months ago

nv-hwoo commented 2 months ago

Add service_kind and endpoint fields to the profile export JSON file:

{
  "experiments": { ... },
  "version": "1.2.3",
  "service_kind": "openai",
  "endpoint": "v1/chat/completions"
}

This allows GenAI-Perf to gather profile metadata through the profile export file and is needed for GenAI-Perf compare subcommand to parse the profile export files directly.

nv-hwoo commented 2 months ago

@debermudez not yet. I am also waiting for CI to pass so will request for review after those are all clear.

debermudez commented 2 months ago

Confirmed offline that the form of the json looks good. LGTM!