search

pwollstadt / IDTxl

The Information Dynamics Toolkit xl (IDTxl) is a comprehensive software package for efficient inference of networks and their node dynamics from multivariate time series data using information theory.
http://pwollstadt.github.io/IDTxl/
GNU General Public License v3.0
249 stars 76 forks source link

Issues in statistical testing #108

Closed daehrlich closed 2 months ago

daehrlich commented 10 months ago

The current version of IDTxl contains issues in the statistical testing.

Please wait for this issue to be marked as fixed before using the affected code features. For more information, see the linked pull request.

JulesMitchell commented 7 months ago

Thanks for your work on the IDTxl package.

Is there any ETA that these bug fixes will be released in version 1.6?

mwibral commented 2 months ago

The bugfix has been merged into master. When testing/reviewing results please be aware that: 1) Our statistical tests in the default setting are hierarchical tests (target level -> source level, see Novelli et al., Network Neuroscience, 2019; https://pubmed.ncbi.nlm.nih.gov/31410382/), keep this fact in mind when interpreting results. 2) FDR correction is not Bonferroni correction - you will typically find more false positives with FDR compared to Bonferroni. 3) The sequential max stats also has a slightly higher false positive rate than max stats.

JulesMitchell commented 1 month ago

Hi Michael,

Thank you, and the team for merging the bugfix. On an unrelated point, I'm attempting to use the 'add_conditionals': 'faes" method, however, I get an IndexError "string index out of range" for the _lag_to_idx function. Must the min source/target lags be set to something specific for this to work?

I appreciate this is not the forum for such a question but I figured it best to post here as the google group hasn't seen any activity lately.

mwibral commented 1 month ago

Dear Jules,

we'll have a look. 

For the Faes method, a zero lag für die conditionals is used. You may be able to add that manually.

Best, Michael

On Wed, 2024-10-16 at 23:08 -0700, JulesMitchell wrote:

Hi Michael, Thank you, and the team for merging the bugfix. On an unrelated point, I'm attempting to use the 'add_conditionals': 'faes" method, however, I get an IndexError "string index out of range" for the _lag_to_idx function. Must the min source/target lags be set to something specific for this to work? I appreciate this is not the forum for such a question but I figured it best to post here as the google group hasn't seen any activity lately. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you modified the open/close state.Message ID: @.***>