NVTabular is a feature engineering and preprocessing library for tabular data designed to quickly and easily manipulate terabyte scale datasets used to train deep learning based recommender systems.
Apache License 2.0
1.05k
stars
143
forks
source link
Slow performance of Categorify operation on Triton Inference Server #1885
When running an NVTabular workflow with Categorify operations in Triton Inference Server, the performance is significantly slow when dealing with high cardinality data.
The Categorify operation should perform efficiently, with category data being cached between requests, resulting in performance similar to that observed in a Jupyter notebook environment.
Actual Behavior
The Categorify operation is slow, with each request taking as long as the first request, suggesting that category data is not being effectively cached between requests.
Results
Below are the result based on benchmarking script - encode.sh
Description
When running an NVTabular workflow with Categorify operations in Triton Inference Server, the performance is significantly slow when dealing with high cardinality data.
Environment
Steps to Reproduce
Expected Behavior
The Categorify operation should perform efficiently, with category data being cached between requests, resulting in performance similar to that observed in a Jupyter notebook environment.
Actual Behavior
The Categorify operation is slow, with each request taking as long as the first request, suggesting that category data is not being effectively cached between requests.
Results
Below are the result based on benchmarking script - encode.sh