Closed lhnwrk closed 3 months ago
We've gotten that fix in. @lhnwrk: could you please rebase this branch on top of master
?
@jesse-c Rebased on master
!
@lhnwrk: Thank you! I've had to re-approve it and run the checks, so they're going now. As soon as it passes, I'll merge it in.
Merged in! Thank you again, @lhnwrk. šš»
Pandas codec currently breaks mlflow runtime's schema enforcement (#1625). This is because numeric mlflow datatypes are assigned
NumpyCodec
content type by default, which decodesPandasCodec
's explicit batch size into a column vector:However,
pd.Series
expects one-dimensional data, and when the_decoded_payload
is converted to apd.Series
here returns unexpected results:This seems like a bug from pandas side,
dtype="int64"
should have thrown an error (and in fact it does for higher dimension, but for some reason is okay with[np.array([1]), np.array([2]), np.array([3])]
and weirdly casts them into tuples), but either way mlserver should have dropped the trailing axis aspd.Series
already implies column vector.From this test, it seems like the conversion to
list
in the codec was to accommodate the use case ofpd.Series
of tensors:I'd argue in the context of Pandas codec this should be made explicit with shape
[1, 1]
and datatypeBYTES
for array elements. In fact, a test above does so, specifying the intended shape is a[2,1]
column vector with datatypeBYTES
forlist
elements.This PR explicitly reshapes
np.array
payload to expected shapes before conversion topd.Series
.