Open issues for triton-inference-server (round 2) - Githubissues

fastmachinelearning / SonicCMS

Services for Optimized Network Inference on Coprocessors (for CMS)

8 stars 8 forks source link

Open issues for triton-inference-server (round 2) #17

Open kpedro88 opened 3 years ago

kpedro88 commented 3 years ago

Tracking the second round of issues submitted to triton-inference-server:

[ ] https://github.com/triton-inference-server/server/issues/2018: Control number of threads used by CPU server
[x] https://github.com/triton-inference-server/server/issues/2019: Stricter model versioning
[ ] https://github.com/triton-inference-server/server/issues/2020: Ability to disable or redirect cout/cerr
[x] https://github.com/triton-inference-server/server/issues/2021: Support input/output compression
[x] https://github.com/triton-inference-server/server/issues/2158: Ragged batching support for ML backends
[ ] https://github.com/triton-inference-server/server/issues/2159: Support different network protocols
[x] https://github.com/triton-inference-server/server/issues/3141: CMake problems for client library in v2.11.0

kpedro88 commented 3 years ago

Model checksum support has been added: https://github.com/triton-inference-server/checksum_repository_agent

kpedro88 commented 3 years ago

I/O compression is available and implemented in CMSSW in https://github.com/cms-sw/cmssw/pull/34508