search

py-why / causal-learn

Causal Discovery in Python. It also includes (conditional) independence tests and score functions.
https://causal-learn.readthedocs.io/en/latest/
MIT License
1.04k stars 174 forks source link

PNL - Test2 #153

Open diogocruz17 opened 7 months ago

diogocruz17 commented 7 months ago

Hello. I was applying the PNL algorithm to some data and was getting unexpected results. Upon running the tests, I get an error in test_pnl_simulation_2, for the p_value_forward_2 value. Is there a known problem regarding this? Screenshot from 2023-11-22 15-41-45 Thank you

ErdunGAO commented 7 months ago

Hello. I was applying the PNL algorithm to some data and was getting unexpected results. Upon running the tests, I get an error in test_pnl_simulation_2, for the p_value_forward_2 value. Is there a known problem regarding this? Screenshot from 2023-11-22 15-41-45 Thank you

Thank you for bringing up this issue. I've run the second test myself, and it successfully passed.

image

I'm considering that the error might be raised from the update in some fundamental packages, such as scipSklearnlearn. Could you verify the precise values of p_value_forward_2 and p_value_backward_2? We're considering updating that asserting exact values to defining intervals, which we think will provide better resilience against potential minor issues arising from future updates to these core packages.

diogocruz17 commented 7 months ago

That seems to be the problem, there is a small difference in the p_value_forward_2 to the simulated one. image Third test passes and the real case also has a small deviation. image

ErdunGAO commented 7 months ago

That seems to be the problem, there is a small difference in the p_value_forward_2 to the simulated one. image Third test passes and the real case also has a small deviation. image

Thanks for the reporting results! We have encountered a similar issue in some other implementations before, where updates to the underlying computation library caused the test examples to fail. Thank you for your information; we will update these assert statements ASAP.