Integrate HugeCTR into the new Triton ensemble API

NVIDIA-Merlin / systems

Merlin Systems provides tools for combining recommendation models with other elements of production recommender systems (like feature stores, nearest neighbor search, and exploration strategies) into end-to-end recommendation pipelines that can be served with Triton Inference Server.

Apache License 2.0

90 stars 30 forks source link

Integrate HugeCTR into the new Triton ensemble API #30

Open karlhigley opened 2 years ago

karlhigley commented 2 years ago

Provide support for serving HugeCTR models in Merlin Systems

viswa-nvidia commented 2 years ago

@karlhigley , which roadmap ticket does this map to ?

karlhigley commented 2 years ago

@viswa-nvidia This never mapped to a roadmap ticket, it was started (and mostly completed) on our own initiative during the stretch of time when the roadmap tickets didn't cover Systems.

viswa-nvidia commented 2 years ago

@oliverholworthy , please link the relevant ticket/PR. Tag the relevant cuDF team in this ticket

viswa-nvidia commented 2 years ago

Blocked: Oliver cannot run on his laptop

oliverholworthy commented 2 years ago

Base operator PR here: https://github.com/NVIDIA-Merlin/systems/pull/129
Issue opened on HugeCTR here to track the issue with segmentation fault https://github.com/NVIDIA-Merlin/HugeCTR/issues/356