Deterministic results when skipping instances

[AnthonyAtSigma2]

Describe the bug When attempting to reduce the number of input values (aka X_test), results start to vary on predictions made by the interval regressor. I'm not entirely sure this is a bug, but I was under the impression that this algorithm should be possible to parallelize by breaking X_test into pieces.

To Reproduce Split your X_test into multiple pieces and attempt to run the explain_factual method.

Expected behavior When combining the pieces, the array should match if I did not break X_test into pieces

Screenshots If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: [e.g. iOS]
• Browser [e.g. chrome, safari]
• Version [e.g. 22]

Additional context I apologize in advance if I am misunderstanding the algorithm and this is not a bug. Any tips would be greatly appreciated.

[AnthonyAtSigma2]

I have determined the cause of the non-deterministic behavior I observed. It turns out that this package uses a class from the crepes library used to predict values with a conformal predictive system. The implementation does not use random_state, and this causes it to produce randomness which leads to slightly different results depending on the size and order of X_test. In my case, I am able to patch around this, but it required me to copy the class from crepes to add a random_state and an alternative call path for generating the random values.

Hope this helps someone in the future, and perhaps this library can be updated to work around this, unless the randomness provided by the predictive system was intentional and desireable.

[tuvelofstrom]

Thanks for pointing this out! It is definitely something that should be possible to do in the way that you intended.

Could you send me an example of the code with your changes made to crepes and our package, then I can work on a solution based on that: tuwe.lofstrom at ju.se