search

SeldonIO / MLServer

An inference server for your machine learning models, including support for multiple frameworks, multi-model serving and more
https://mlserver.readthedocs.io/en/latest/
Apache License 2.0
705 stars 180 forks source link

The `isnan` method doesn't work on Numpy arrays with non-numeric #1873

Open baifachuan opened 2 months ago

baifachuan commented 2 months ago

My DataFrame has a column with list type, but the value is None, when I use: 

tools: list = None
df=pd.Series(
            [
                tools
            ],
            index=[     
                "tools"
            ],
        )
PandasCodec.encode_request(df, use_bytes=False)

Always found this errors if mlserver >= 1.4.0:

TypeError: ufunc 'isnan' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''

I found the code:

https://github.com/SeldonIO/MLServer/blob/release/1.4.0/mlserver/codecs/numpy.py#L108

How can I make it work correctly on versions above 1.4.0?

ramonpzg commented 2 months ago

Hi @baifachuan -- Thanks for bringing this up. I can see the kind of error you are getting and I can also replicate it locally. I am having difficulties understanding your use case though, would you please describe what you are trying to do?

baifachuan commented 2 months ago

Hi @baifachuan -- Thanks for bringing this up. I can see the kind of error you are getting and I can also replicate it locally. I am having difficulties understanding your use case though, would you please describe what you are trying to do?

Hi @ramonpzg thanks for your replay, I deployed a locally privatized large language model (LLM) through Seldon.

These parameters need to be passed to Seldon when chat with LLM. 

df=pd.Series(
            [
                tools
            ],
            index=[     
                "tools"
            ],
        )

df is the chat parameters, because tools is LLM TOOL list, So they definitely won't be of a numerical type.

bwallima commented 3 days ago

Is there any update regarding that bug? I am experiencing the same issue. This problem occurs as soon as there is an attempt to pass any non-numeric values. For instance, I have a dfthat contains text values, some of which can be None. A fix would be to check for float values in mlserver/codecs/numpy.py row 109: if isinstance(val, float) and np.isnan(val)