MI values when cholesky failed

EtienneCmb commented 4 years ago

@nvoges @brovelli

The current version of the code failed on Nicoles' simulated data, because of the cholesky decomposition when the data are redundant.

@brovelli suggested np.clip(mi, 0, mi). This avoid negative MI values (I think it should also works on negative infinite). @nvoges do you get infinite values or NaN values? Because if NaN, the clip is not going to make a difference. In addition, I've seen negative MI because of the bias correction so I'm not sure it's a good idea to put a hard threshold at 0, especially because we also compute permutations and I'm afraid of having many permutations to zero because of that.

What do u think?

brovelli commented 4 years ago

You are right, np.clip(mi, 0, mi) cannot work for biased corrected GCMI values.

BUT, you can make the np.clip(mi, 0, mi) before computing the bias corrected values

EtienneCmb commented 4 years ago

Ok, it should works indeed. But @nvoges should clarify if she gets infinite or nan values before fixing it

nvoges commented 4 years ago

hi & sorry for the late reply, but I received this mail on my old Juelich account... I'll try to find and change these settings in git.

My initial problem was that I received an error (LinAlg error) and no values at all.

If I "force" the code to work (e.g. by including a selected set of nodes) then I can get unreasonable high values (MI~21) but neither nan nor infinite.

Maybe you are refering to other values, but which ones?

Von: Etienne Combrisson [notifications@github.com] Gesendet: Montag, 16. März 2020 16:04 An: brainets/frites Cc: Voges, Nicole; Mention Betreff: Re: [brainets/frites] MI values when cholesky failed (#2)

Ok, it should works indeed. But @nvogeshttps://github.com/nvoges should clarify if she gets infinite or nan values before fixing it

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/brainets/frites/issues/2#issuecomment-599586269, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AD35DMP4E5TKNJJ72TFHVX3RHY5XHANCNFSM4LMIPFWA.

Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Volker Rieke Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt

EtienneCmb commented 4 years ago

Ok, in that case, the np.clip is not going to fix your issue. One possibility :

try:
    chcx = np.linalg.cholesky(cx)
except:
    return 0

If the cholesky decomposition failed, the returned MI is set to 0. But I'm hesitating of putting it in the gcmi_nd.py functions because I'm afraid of having new silent errors with MI and permuted MI set to zero because it silently fail...

I'm really not sure this should be included in Frites, I think it might be dangerous for us

EtienneCmb commented 4 years ago

Maybe with an additional error message...

@brovelli what do you think?

brainets / frites

MI values when cholesky failed #2