search

triton-inference-server / fastertransformer_backend

BSD 3-Clause "New" or "Revised" License
411 stars 133 forks source link

Config.pbtxt for all_models/t5/fastertransformer incorrect #73

Open dhaval24 opened 1 year ago

dhaval24 commented 1 year ago

Description

The Latest faster transformer v5.1.1 which is being used by the Fastertransformer backend latest release prescribes that T5 decoder output - [output_ids and sequence_length] should be int32 type however in the current config.pbtxt its specified as uint32.

https://github.com/NVIDIA/FasterTransformer/blob/release/v5.1.1_tag/docs/t5_guide.md

Please update the config.pbtxt without correct output configurations.

Reproduced Steps

Run T5 in ensemble mode and the post processing logic will indicate that the output should be INT32.
byshiue commented 1 year ago

We have updated the config.pbtxt of latest v1.4 version.