R1 method using default parameters fail with error 'problem too large'

BergmannLab / MONET

MONET : MOdularising NEtwork Toolbox - https://doi.org/10.1093/bioinformatics/btaa236

GNU General Public License v3.0

41 stars 15 forks source link

R1 method using default parameters fail with error 'problem too large' #15

Open cristian931 opened 4 years ago

cristian931 commented 4 years ago

The K1 and M1 method with the same input gave no problem. I'm working on a Centos7 machine with 82GB of RAM, so the memory should not be the issue. I'm using singularity as container.

This is the command line: ~/MONET/monet --input=network_retinal_toxicity_BIANA.txt --method=R1 --container=singularity

The log file reports as follow 2020-02-19-101559R1console-output__network_retinal_toxicity_BIANA.txt

This is the input file for reproducing: It's an undirected network file formatted as explained in the readme file, since it's not weighted the weights have been set equal to 1 for every edge (don't know if this may be the problem, but again with the other two methods there were no problems). The nodes are identified by a numerical code. The total number of edges is around 400. network_retinal_toxicity_BIANA.txt

For the moment i want to use the standard parameters for later try to tune it based on the results.

Thanks for the help

mattiat commented 4 years ago

Thank you so much, Cristian! I reproduced today. The input seems well formatted.

mattiat commented 4 years ago

I added a system test to reproduce this: MONET/.test/system_test/retinal_toxicity_test.sh

WeijiaZhang24 commented 4 years ago

Hi all, This was caused by converting sparse matrix into matrix. It has been addressed now and passed the test.

mattiat commented 4 years ago

I confirm the test passes.

mattiat commented 4 years ago

I am re-running, but there seems to be something weird with the number of edges.

Cristian931 says "number od edges around 400" (indeed input has 393 lines)
R1 debugging text indicates around 40000

@WeijiaZhang24: am I getting this wrong?

WeijiaZhang24 commented 4 years ago

I think this is caused by how the underlying C code (which is kinda difficult to address in a short time) handles input where nodes with no edge connected. It doesn't affect the result.

mattiat commented 4 years ago

PROBLEM: this fix causes system tests to fail. Reverting changes. To be fixed.

mattiat commented 4 years ago

@WeijiaZhang24 Please continue to work here (and sorry for merging too early , my bad!) https://github.com/BergmannLab/MONET/commits/issues_15.

Make sure the following pass:

MONET/.test/system_test/retinal_toxicity_test.sh
MONET/.test/system_test/zachary_karate_club_test.sh
MONET/.test/system_test/reproduce_challenge/reproduce_challenge.sh

WeijiaZhang24 commented 4 years ago

@mattiat I pushed a new branch "issues_15_2", which used exception handling in R to fix the problem. Tested with the 3 scripts it seems fine. Sorry for this messy patch, I will try to find a more elegant solution when I can squeeze more time to work on this.

mattiat commented 4 years ago

@WeijiaZhang24 Thank you! The last system test still fails for me, I attach the error file 2020-03-10-132648R1console-output__6_homology_anonym_v2.txt