search

bcbio / bcbio-nextgen

Validated, scalable, community developed variant calling, RNA-seq and small RNA analysis
https://bcbio-nextgen.readthedocs.io
MIT License
994 stars 353 forks source link

Does bcbio support smooth restart when a job is stopped? #3717

Closed lovebaboon1989 closed 1 year ago

lovebaboon1989 commented 1 year ago

Hi there, I'm running bcbio-nextgen on our institution's cluster server, since I have many samples for bulkRNA processing (fastq files to gene count matrix) but the usage duration for one job on the cluster is limited (5 days), I just wonder if bcbio can continue the pipeline smoothly after one job is exited and restarted after 5 days? Many thanks!

sjhosui commented 1 year ago

Hi,

Yes, bcbio-nextgen tracks provenance so it can see what was most recently generated and resume from that point.

Depending on how your cluster is set up, you might also consider splitting your bulk RNA-seq samples into smaller batches for processing and then merging the count matrices at the end.

lovebaboon1989 commented 1 year ago

Hi, thanks a lot for the quick reply, that helps really a lot!