triton-inference-server / server

The Triton Inference Server provides an optimized cloud and edge inferencing solution.
https://docs.nvidia.com/deeplearning/triton-inference-server/user-guide/docs/index.html
BSD 3-Clause "New" or "Revised" License
8.4k stars 1.49k forks source link

refactor: Removing `Server` subclass from `tritonfrontend` #7683

Closed KrishnanPrash closed 1 month ago

KrishnanPrash commented 1 month ago

What does the PR do?

Removes the keyword Server from the tritonfrontend package to avoid confusion with multiple definitions of Server in the core and frontend bindings.

Previously, using tritonfrontend.KServeHttp would look like this:

import tritonserver
from tritonfrontend import KServeHttp, KServeGrpc

server = tritonserver.Server(...).start(wait_until_ready=True)
http_service = KServeHttp.Server(server)
http_service.start()

New workflow replaces KServeHttp.Server with just KServeHttp:

import tritonserver
from tritonfrontend import KServeHttp, KServeGrpc

server = tritonserver.Server(...).start(wait_until_ready=True)
http_service = KServeHttp(server)
http_service.start()