I have recently been running your DeepHiC model on some new data and ran into some memory issues (~600Gb) when preprocessing the HiCPro output format.
It seems that the reason for this was converting the sparse contact matrix to dense matrix prematurely. By using sparse matrix operations all the way through I think you can significantly save memory and time.
omegahh
commented
3 years ago
Hi,
I have recently been running your DeepHiC model on some new data and ran into some memory issues (~600Gb) when preprocessing the HiCPro output format.
It seems that the reason for this was converting the sparse contact matrix to dense matrix prematurely. By using sparse matrix operations all the way through I think you can significantly save memory and time.
Thank you for providing your code and models!