corrects forward incompatibility caused by ModelInput(**model_input) invocation
more cleanly carves out type conversion for oai
provides temporary fix for trt llm "stream=False" shortcomings, that results in generating nothing (cc @htrivedi99 @aspctu )
:computer: How
:microscope: Testing
Deployed this on a Llama-3-8b-trt-llm model and ran it through its paces (stream=True, False), additional params to ensure forward compatibility, printed model input from truss to verify functionality
:rocket: What
Addresses some shortcomings in the trtllm exmaple
ModelInput(**model_input)
invocation:computer: How
:microscope: Testing
Deployed this on a Llama-3-8b-trt-llm model and ran it through its paces (stream=True, False), additional params to ensure forward compatibility, printed model input from truss to verify functionality
Some Follow Ups: