search

triton-inference-server / client

Triton Python, C++ and Java client libraries, and GRPC-generated client examples for go, java and scala.
BSD 3-Clause "New" or "Revised" License
521 stars 225 forks source link

Support Python 3.8 in GenAI-Perf #643

Closed dyastremsky closed 2 months ago

dyastremsky commented 2 months ago

GenAI-Perf says it supports Python 3.8. This pull request fixes the breaking Python 3.8 code (i.e. removes features only available in 3.9+). To confirm it continues to support Python 3.8, it adds a job to build and run testing against 3.8 for each PR.

A few retrofits for Python 3.8:

  1. Using the typing library rather than the built-in subscriptable types.
  2. Using Union rather than "|"
  3. Using a custom pairwise function rather than the itertools function
  4. Using custom logic rather than the in-built removeprefix function
lkomali commented 2 months ago

LGTM

dyastremsky commented 2 months ago

Thanks, Harshini! Just a note to everyone that this PR still needs approval. :)

debermudez commented 2 months ago

1 sec, before we merge. Is this an update we need? do we have a customer that is asking for 3.8 support?

dyastremsky commented 2 months ago

1 sec, before we merge. Is this an update we need? do we have a customer that is asking for 3.8 support?

This is related to our offline conversation. It sounded like there was agreement (and approval from Ganesh) for adding it. GenAI-Perf currently says it supports Python 3.8.

PyTriton also supports 3.8+ and Triton CLI did as well before it was dropped until GenAI-Perf supports 3.8+ as well. We either need to drop 3.8-3.9 support in our documentation or support and test against it.

Please refer to the offline conversation as it has the back-and-forth decision making behind this pull request.

debermudez commented 2 months ago

1 sec, before we merge. Is this an update we need? do we have a customer that is asking for 3.8 support?

This is related to our offline conversation. It sounded like there was agreement (and approval from Ganesh) for adding it. GenAI-Perf currently says it supports Python 3.8.

PyTriton also supports 3.8+ and Triton CLI did as well before it was dropped until GenAI-Perf supports 3.8+ as well. We either need to drop 3.8-3.9 support in our documentation or support and test against it.

Please refer to the offline conversation as it has the back-and-forth decision making behind this pull request.

Got it. I forgot about that convo. You are covering all the bases and this is good to go forward. Thanks for explaining it to me.