Is there any support for text preprocessing mainly for transformer models

triton-inference-server / dali_backend

The Triton backend that allows running GPU-accelerated data pre-processing pipelines implemented in DALI's python API.

https://docs.nvidia.com/deeplearning/dali/user-guide/docs/index.html

MIT License

123 stars 29 forks source link

Is there any support for text preprocessing mainly for transformer models #86

Open sachinsharma9780 opened 3 years ago

sachinsharma9780 commented 3 years ago

Hi,

I have seen that dali already have prebuilt functions for image preprocessing like dali.fn.resize(images, resize_x=299, resize_y=299). But does it provide any preprocessing functions for text as well like performing tokenization of text on the Triton server-side, typically for transformer models?

Thanks!

szalpal commented 3 years ago

Hello @sachinsharma9780 !

Unfortunately, at the moment DALI doesn't support text preprocessing. Is there any particular tokenizer you are referring to? As far as I know, transferring tokenization on the GPU wouldn't make much gain, since this algorithm is not easily parallelizeable.