Open stikkireddy opened 2 months ago
Ray Serve is a phenomenal serving engine that abstracts serving and some throughput optimization features like batching, async execution, pipelining, etc. Supports torch and other popular frameworks. This can be used for the following models:
some common embedding models:
clip-ViT-B-32
dicta-il/dictabert-joint
almanach/camembert-base
Ray Serve is a phenomenal serving engine that abstracts serving and some throughput optimization features like batching, async execution, pipelining, etc. Supports torch and other popular frameworks. This can be used for the following models: