Download ML models during deploy

[lesters]

The current procedure for using/importing ML models to Vespa is to put them in the application package under the models directory (see https://docs.vespa.ai/documentation/reference/application-packages-reference.html). For many applications it would be more convenient to refer to the model by URL, and have Vespa download and convert the model during deployment. This is not handled by URL downloading (https://docs.vespa.ai/documentation/configuring-components.html#adding-files-to-the-component-configuration), as that handles files to be used in custom components. In contrast, this feature should download model files and have them converted to ranking expressions and constant tensors, for instance by downloading to the models directory before models are converted. Details such as how to configure and refer to the model in the search definition must also be handled.

This would cater to the use case where a model is (periodically) trained on some external system and requires a deploy to activate the models, for instance for XGBoost models or TensorFlow/ONNX models where model architecture can change. This differs from the case where tensors are updated without redeploy (https://docs.vespa.ai/documentation/tensorflow.html#updating-variables-without-redeploying-the-application), as that requires a fixed set of tensors to update.

[prasad-marne]

Any updates on this? we right now have to do this in our CD flow. For bigger models it would be better for vespa to take care of it.

[bratseth]

Are these models used in ranking expressions, or by components (e.g models for embedding)?

[prasad-marne]

By components for embedding generation

[bratseth]

Then you can use an url now, see https://docs.vespa.ai/en/configuring-components.html#adding-files-to-the-component-configuration