some help running degNorm

[dllahr]

Hi

First, thank you for providing the degNorm tool (and writing the paper describing it) - it seems like a great resource - very much appreciated.

I'm writing to ask for some help / guidance in running the tool (details of setup / command below.). I'd like to get a sense of if I'm heading in the right direction or not, a rough sense of the compute required to run, any guidance would be very helpful and much appreciated:

• I ran the command with 9 samples, it has been running for ~10 hours, it only appears to have worked on 1 sample (based on output files) - is this normal?
  • I used -p 8 option, it appears to have used ~3 CPUs briefly, then ~1.6 CPU for 5 hours, has been using 1 CPU for the last 5 hours
• there has been no output written for the last 5 hours - either to the log file or the output directory - output for only the one sample is present - is this expected?
• memory usage maximum is ~10 GB, ~6.7 GB on average - is this typical?
• I was thinking as speed enhancement I could run the prewarm step as separate degnorm jobs, so each sample could run in parallel - is there a way to run just the prewarm step?
  • Otherwise I could run pairs of samples as separate jobs - do you recommend this?
• it detected our samples as single end, but we ran them as paired end - I'm not sure what to make of this

Here's the command I used to run:

nohup degnorm --bam-files $(cat input_bam.grp ) --bai-files $(cat input_bai.grp ) \
-g gencode.v27.primary_assembly.annotation.gtf \
-p 8 -o output &> degnorm.log &

I'm using an AWS EC2 instance t3a.2xlarge (8 CPUs, 32 G RAM) running AWS linux (CentOS variant). Installation of degnorm was normal, tests passed.

Any ideas / suggestions would be much appreciated - thank you in advance, Dave

[ffineis]

Hi Dave,

Apologies that the program has been slow on your 9 samples. The python CLI tool is memory inefficient, and does not make proper use of multiprocessing during the NMF routine, leading to long processing times. I am no longer actively maintaining this repo and highly recommend that you switch to using the more optimized R library.

I will leave an archive notice on the python package's README file this week with plans to archive in March 2023.

Thanks, Frank

[dllahr]

Thank you for the response - much appreciated. I'll try that out.