vitrivr / vitrivr-engine

vitrivr's next-generation retrieval engine. It is capable of extracting and retrieving a wider range of multimedia objects such as audio, video, images or 3d models.
https://vitrivr.org
MIT License
6 stars 3 forks source link

[Feature] TorchServe Integration #122

Closed ppanopticon closed 15 hours ago

ppanopticon commented 4 days ago

This PR provides features based on TorchServe. It consists of the following components:

A TSImageLabel feature can be configured as follows:

"fields": {
 "torchserve": {
  "factory": "TSImageLabel",
   "parameters": {
    "host": "127.0.0.1",
    "port": "7070",
    "model": "densenet161",
    "token": "099-Ed4q",
    "threshold": "0.5"
  }
 }
}

Important: It is up-to the user to deploy the model in TorchServe beforehand. The following parameters exist for configuration (the first four are used irrespective of the model that is being used):

Parameter Description
host Host of the TorchServe server.
port Port of the TorchServe server (gRPC endpoint).
token Authentication token for the inference endpoint (optional)
model Name of a deployed model.
threshold Confidence threshold for labels (for TSImageLabel only).

For now, the PR is a draft to allow for discussion and optimisation.

ppanopticon commented 4 days ago

The retrieval side of things is currently missing.

Main reason is, that I'm not quite sure how we would use a LabelDescriptor for querying. Is there any existing facility, that does that? I couldn't find any. From where I stand, LabelDescriptors could be used both for classical boolean retrieval.

ppanopticon commented 19 hours ago

Added a (preliminary) version of the retrieval side and addressed @lucaro's comments.