search

Kipok / NeMo-Skills

A pipeline to improve skills of large language models
https://kipok.github.io/NeMo-Skills/
Apache License 2.0
185 stars 41 forks source link

Fix fill output args #86

Closed yinglin-nv closed 3 months ago

yinglin-nv commented 3 months ago

With the change in this recently merged PR, batch_input_ids_list is now required by the _fill_output() function in ModelCppRunner. In this PR, we updated serve_trt.py accordingly by passing batch_input_ids_list to runner._fill_output() in _stream().

Kipok commented 3 months ago

Thanks @yinglin-nv! I ported this to #87 along with some other changes related to upgrade, so let's close this one.