search

SeldonIO / MLServer

An inference server for your machine learning models, including support for multiple frameworks, multi-model serving and more
https://mlserver.readthedocs.io/en/latest/
Apache License 2.0
659 stars 179 forks source link

MLFlow model with Array type in signature not loading #1802

Open wbinek opened 3 weeks ago

wbinek commented 3 weeks ago

I was trying to serve a model in MLFlow Model format created using MLFlow > 2.10. The MLFLow 2.10 added support for Array and Object datatypes in signature (https://www.mlflow.org/docs/latest/model/signatures.html), which seems to be unsupported by the MLServer.

Error:

2024-06-12 08:10:45,410 [mlserver.parallel] ERROR - An error occurred processing a model update of type 'Load'.
Traceback (most recent call last):
  File "/opt/conda/lib/python3.10/site-packages/mlserver/parallel/worker.py", line 158, in _process_model_update
    await self._model_registry.load(model_settings)
  File "/opt/conda/lib/python3.10/site-packages/mlserver/registry.py", line 293, in load
    return await self._models[model_settings.name].load(model_settings)
  File "/opt/conda/lib/python3.10/site-packages/mlserver/registry.py", line 148, in load
    await self._load_model(new_model)
  File "/opt/conda/lib/python3.10/site-packages/mlserver/registry.py", line 165, in _load_model
    model.ready = await model.load()
  File "/opt/conda/lib/python3.10/site-packages/mlserver_mlflow/runtime.py", line 159, in load
    self._sync_metadata()
  File "/opt/conda/lib/python3.10/site-packages/mlserver_mlflow/runtime.py", line 171, in _sync_metadata
    self.inputs = to_metadata_tensors(
  File "/opt/conda/lib/python3.10/site-packages/mlserver_mlflow/metadata.py", line 57, in to_metadata_tensors
    datatype, content_type = _get_content_type(input_spec)
  File "/opt/conda/lib/python3.10/site-packages/mlserver_mlflow/metadata.py", line 41, in _get_content_type
    return _MLflowToContentType[input_spec.type]
TypeError: unhashable type: 'Array'
2024-06-12 08:10:45,411 [mlserver] INFO - Couldn't load model 'mlflow-model'. Model will be removed from registry.

MLFlow Model Signature:

signature:
  inputs: '[{"type": "datetime", "name": "start", "required": true}, 
            {"type": "array", "items": {"type": "array", "items": {"type": "long"}}, "name": "target", "required": true}, 
            {"type": "array", "items": {"type": "string"}, "name": "feat_static_cat", "required": true}, 
            {"type": "array", "items": {"type": "array", "items": {"type": "boolean"}}, "name": "feat_dynamic_real", "required": true}]'
  outputs: null
  params: null