Enable cpp code path for `Categorify` ops

NVIDIA-Merlin / systems

Merlin Systems provides tools for combining recommendation models with other elements of production recommender systems (like feature stores, nearest neighbor search, and exploration strategies) into end-to-end recommendation pipelines that can be served with Triton Inference Server.

Apache License 2.0

90 stars 30 forks source link

Enable cpp code path for `Categorify` ops #389

Closed rjzamora closed 4 months ago

rjzamora commented 6 months ago

Rolls back one of the changes in https://github.com/NVIDIA-Merlin/systems/pull/354 to improve inference performance of Categorify.

github-actions[bot] commented 6 months ago

Documentation preview

https://nvidia-merlin.github.io/systems/review/pr-389

oliverholworthy commented 6 months ago

the docs related errors are now resolved.

There's two remaining things for this MR:

fix the lint errors (reformat with black pre-commit run --all-files black) and the codespell error (typo in everything)
merge the NVTabular PR https://github.com/NVIDIA-Merlin/NVTabular/pull/1874
- currently blocked by some unrelated errors with docs (which will be resolved by https://github.com/NVIDIA-Merlin/NVTabular/pull/1878 )

rjzamora commented 6 months ago

There's two remaining things for this MR: ...

Thanks so much @oliverholworthy ! Sorry, I missed this earlier.

oliverholworthy commented 6 months ago

Following the merge of https://github.com/NVIDIA-Merlin/NVTabular/pull/1874 we're now getting a new error in the tests

RuntimeError: Error: <class 'ValueError'> - Unknown column for CategorifyTransform x__values

rjzamora commented 5 months ago

I haven't had the time to understand the test failure well enough to work through a long-term fix. However, my last commit roles back the decision to use the cpp code path for "Categorify" by default. Instead, when the WorkflowRunner is initialized, the value of the NVT_CPP_OPS environment variable will dictate which NVT ops the cpp code path will be used for.

For example, you can enable the cpp version of Categorify by deploying a triton model as follows:

NVT_CPP_OPS="Categorify" tritonserver --model-repository=/my-model/

jperez999 commented 4 months ago

This is failing right now on some package mismatches right now. A solution is to update the tritonclient package to 2.30.0. Working on that solution now. Then we should be able to pick up the newest version of CI container to fix the triton errors.