triton-inference-server / server

The Triton Inference Server provides an optimized cloud and edge inferencing solution.
https://docs.nvidia.com/deeplearning/triton-inference-server/user-guide/docs/index.html
BSD 3-Clause "New" or "Revised" License
8.37k stars 1.49k forks source link

refactor: moving `tritonfrontend` to `@handle_triton_error` decorator #7720

Closed KrishnanPrash closed 1 month ago

KrishnanPrash commented 1 month ago

What does the PR do?

Abstracts error handling in the tritonfrontend/_api away with the decorator @handle_triton_error. This removes the need to wrap every function in _api/_<service>.py with a try: ... catch: ... manually.

Side Note for the dunder functions: For _api/_<service>.py::__enter__(): Since this method just calls start() and start() has @handle_triton_error, any error that occurs in __enter__() will be handled by the desired try: ... catch: .... For _api/_<service>.py::__exit__(self, exc_type, exc_value, traceback): If an exception occurs when inside the context, it will gracefully call exit. But to be captured by @handle_triton_error it must be thrown again.

Where should the reviewer start?

Look at src/python/tritonfrontend/_api/_error_mapping.py

Test plan:

Ran with existing test cases L0_python_api to verify that errors are being handled identically.