Predict in Datapoint-Editor leads to non-deterministic behavior

[LinusEdelkott]

I stumled upon the quite annoying issue, that predicting via the data-point editor returns non-deterministic results.

I created the widget with a custom_predict_function as there's a sklearn-pipeline around my xgbm model. The creation of the widget works fine, but if I change one datapoint in the widgets editor and click on predict and change it back, it results in seemingly random results.

def custom_predict_processed(examples_to_infer):
  model_ins = pd.DataFrame(examples_to_infer)
  preds = model[1].predict(model_ins[model_ins.columns[:-1]])
  return np.concatenate(([preds], [1 - preds]), axis=0).transpose()

config_builder = WitConfigBuilder(proto_df_processed[0:num_datapoints]).set_custom_predict_fn(custom_predict_processed)
widget = WitWidget(config_builder, height=tool_height_in_px) 

On a colleague machine (mac) the code worked fine. I tried two different machines but run into the issue both times.

By simply putting a print into the custom_predict function, I could confirm that it is seemingly not used via the data-point-editor predict.

[jameswex]

What machines did you try it on where it failed and where it succeeded?

Can you provide an exact set of repro steps with a minimal example? The minimal example could use a custom prediction function that returns a number directly from the input example so that we take the actual ML model out of the equation, in terms of what is going wrong.

I just ran a test in colab witih a custom prediction function that logs the calls and it seems to be being called correctly every time the "predict" button is pressed after a datapoint edit.