search

deepjavalibrary / djl-serving

A universal scalable machine learning model deployment solution
Apache License 2.0
182 stars 59 forks source link

[python] log exception stacktrace for exceptions in python, improve r… #2142

Closed siddvenk closed 1 day ago

siddvenk commented 2 days ago

…eturned error message on KeyErrors during parse input

Two changes:

  1. Log the stacktrace for exceptions. In some cases where we log in the except blocks, we don't log any details about the exception/stacktrace making it hard to debug and understand what went wrong.
  2. Improve both the logging and error message for Parse input failures.

Prior to this change, for a parse input failure we would log:

INFO  PyProcess W-267-0a3e93accd53998-stdout: [1,0]<stdout>:Parse input failed: 0

And return to the client:

{'error': "'prompt'", 'code': 424}

Now, for a parse input failure we will log:

INFO  PyProcess W-9340-model-stdout: [1,3]<stdout>:Input Parsing failed. Ensure that the request payload is valid. Invalid Request Property: 'prompt'
INFO  PyProcess W-9340-model-stdout: [1,3]<stdout>:Traceback (most recent call last):
INFO  PyProcess W-9340-model-stdout: [1,3]<stdout>:  File "/tmp/.djl.ai/python/0.29.0-SNAPSHOT/djl_python/utils.py", line 68, in parse_input_with_formatter
INFO  PyProcess W-9340-model-stdout: [1,3]<stdout>:    _inputs, _param, is_client_side_batch[i] = _parse_inputs_params(
INFO  PyProcess W-9340-model-stdout: [1,3]<stdout>:  File "/tmp/.djl.ai/python/0.29.0-SNAPSHOT/djl_python/utils.py", line 117, in _parse_inputs_params
INFO  PyProcess W-9340-model-stdout: [1,3]<stdout>:    _inputs, _param = parse_3p_request(
INFO  PyProcess W-9340-model-stdout: [1,3]<stdout>:  File "/tmp/.djl.ai/python/0.29.0-SNAPSHOT/djl_python/three_p/three_p_utils.py", line 24, in parse_3p_request
INFO  PyProcess W-9340-model-stdout: [1,3]<stdout>:    _inputs = input_map.pop("prompt")
INFO  PyProcess W-9340-model-stdout: [1,3]<stdout>:KeyError: 'prompt'

And return to the client:

{'error': "Input Parsing failed. Ensure that the request payload is valid. Invalid Request Property: 'prompt'", 'code': 424}