search

triton-inference-server / fastertransformer_backend

BSD 3-Clause "New" or "Revised" License
411 stars 133 forks source link

what is the use of preprocessing & postprossing ? can i start fastertransformer only for bloom model ? #168

Open flyingjohn opened 11 months ago

flyingjohn commented 11 months ago

Description

i fail to start preprossing and postprossing. so i start fastertransformer only, it works fine but the model performance is bad. so i wonder if the reason is the missing of preprossing and postprossing ?? what is the use of preprocessing & postprossing ??

Reproduced Steps

/tritonserver/bin/tritonserver --model-repository=triton-model-store/bloom/
yeahdongcn commented 11 months ago

preprocessing: string -> encoded input_ids bloom (fastertransformer): encoded input_ids -> encoded output_ids postprocessing: encoded output_ids -> string

This could be fine if there is no need to run an end-to-end case.