Basic MPI support - Githubissues

triton-inference-server / triton_cli

Triton CLI is an open source command line interface that enables users to create, deploy, and profile models served by the Triton Inference Server.

48 stars 2 forks source link

Closed fpetrini15 closed 10 months ago

fpetrini15 commented 10 months ago

Simple scheme to run trtllm models using mpi. World size is assumed to be 1.

rmccorm4 commented 10 months ago

LGTM other than minor comment

rmccorm4 commented 10 months ago

Beautiful thanks :bow: