search

rh-aiservices-bu / fraud-detection

https://rh-aiservices-bu.github.io/fraud-detection/
Apache License 2.0
12 stars 22 forks source link

Use scaler when sending requests #36

Closed RHRolun closed 1 week ago

RHRolun commented 2 weeks ago

We didn't use the scaler when calculating the prediction from the inference endpoints, which caused the local predictions in notebook 1 different from the request notebooks. This PR aims to fix that.

adelton commented 1 week ago

Tested with OpenShift AI 2.9.1 and I confirm that I now get

Is Sally's transaction predicted to be fraudulent? (true = YES, false = NO) 
False
How likely was Sally's transaction to be fraudulent? 
0.00027%

on the first notebook and

[0.00027432802]
not fraud

on the third one.

Thank you!

cfchase commented 1 week ago

Great catch.

cfchase commented 1 week ago

@RHRolun Do we need to change threshold in notebook 1_experiment_train.ipynb here?: https://github.com/rh-aiservices-bu/fraud-detection/blob/main/1_experiment_train.ipynb?short_path=6950d32#L276

sess = rt.InferenceSession("models/fraud/1/model.onnx", providers=rt.get_available_providers())
input_name = sess.get_inputs()[0].name
output_name = sess.get_outputs()[0].name
y_pred_temp = sess.run([output_name], {input_name: scaler.transform(X_test.values).astype(np.float32)}) 
y_pred_temp = np.asarray(np.squeeze(y_pred_temp[0]))
threshold = 0.995
y_pred = np.where(y_pred_temp > threshold, 1,0)
RHRolun commented 1 week ago

Ah I missed that, good catch, pushing an update

RHRolun commented 1 week ago

@cfchase update the threshold adjusted