search

pwollstadt / IDTxl

The Information Dynamics Toolkit xl (IDTxl) is a comprehensive software package for efficient inference of networks and their node dynamics from multivariate time series data using information theory.
http://pwollstadt.github.io/IDTxl/
GNU General Public License v3.0
237 stars 76 forks source link

pval constant for the whole network #84

Closed lcastri closed 2 years ago

lcastri commented 2 years ago

Hi,

I am using the MultivariateTE network to select features in my dataset based on TE. The algorithm selects the features but then, in order to filter them, I need the p-values. Why do I get always the same value of p-value (0.002) for each relation that the network analyse?

Attached the network's settings and an example of result (target 4).

Thank you Luca

network_analysis = MultivariateTE()
settings = {'cmi_estimator': 'JidtKraskovCMI',
                  'max_lag_sources': 1,
                  'min_lag_sources': 1,
                  'history_target': 1}

result-target4

pwollstadt commented 2 years ago

Hi @lcastri, these constant p-values look like every detected link was highly significant in the permutation test. In the test, the original estimate is compared against the distribution of estimates from shuffled data. The p-value is then the fraction of estimates from shuffled data that are larger than the original value. If no estimate from shuffled data is larger than the original estimate, the p-value is set to 1/no. permutations. This seems to have happened for all estimates in your case, i.e., the p-value has been set to 1/500 for every estimate.

I will close this since this is not an issue with the code. If you have further questions, please move these to the mailing list. If you find anything wrong with the code, please open another issue here.