search

vitrivr / vitrivr-engine

vitrivr's next-generation retrieval engine. It is capable of extracting and retrieving a wider range of multimedia objects such as audio, video, images or 3d models.
https://vitrivr.org
MIT License
5 stars 3 forks source link

Move pytorch-based models over to TorchServe #19

Open lucaro opened 7 months ago

lucaro commented 7 months ago

Whenever we use external models for feature extraction that are based on torch, it's presumably much more efficient to serve them through TorchServe.

ppanopticon commented 5 months ago

So the tasks here is to create an abstract implementation of an Analyser (and its Retriever and Extractor) that can be used with TorchServer?

lucaro commented 5 months ago

Pretty much, yes. TorchServe also has a gRPC API that might be more efficient than having to do things via REST. I have not yet looked into the details of it though.