replicate / cog-triton

A cog implementation of Nvidia's Triton server

Apache License 2.0

11 stars 0 forks source link

Closed yorickvP closed 3 months ago

yorickvP commented 3 months ago

33 contains some changes that make life better and has some further size optimizations.

pushes to triton-base-sm80, triton-base-sm90
remove libnvinfer_builder_resource.so from runners
double the GHA cores
put backends in a separate package, so we can share the nvidia-pytriton package across builds