automl-classroom / iML-ws21-ex08

0 stars 0 forks source link

Different counterfactual values but correct prediction result #3

Closed SwiftPredator closed 2 years ago

SwiftPredator commented 2 years ago

Our create_counterfactual method returns slightly different values than expected by the test cases. However, the prediction is the correct class zero for these values. Can that be traced back to the Rashomon effect (despite setting the seed)? All other test cases are successful, which leads to the assumption that our loss calculation should be correct.

Output from the script:

Original Instance: area perimeter compactness length width asymmetry groove 0 0.321058 0.293388 1.0 0.123874 0.536707 0.591915 0.129

Prediction: 1

Counterfactual for desired class 0, found after 53 steps: area perimeter compactness length width asymmetry groove 0 0.302608 0.259556 1.013048 0.090561 0.505112 0.654258 0.148638

Prediction: 0

maxidl commented 2 years ago

Hi, I took a look at your code and while the resulting counterfactual instance seems ok, the test case fails because an important step in your optimization loop is missing. You can read about it here. If the problem still occurs afterwards, let me know.