10XGenomics / cellranger

10x Genomics Single Cell Analysis
https://www.10xgenomics.com/support/software/cell-ranger
Other
348 stars 92 forks source link

Pipestance failed. Use --noexit option to keep UI running after failure. #89

Closed zoe106 closed 3 years ago

zoe106 commented 4 years ago

Hi, my command is as followed: nohup time cellranger count --id=SRX6991953 \ --sample=SRX6991953 \ --fastqs=/xtdisk/apod-gen/haoll/xiongyj/1-10X/PRJNA577618/1-output/SRX6991953/ \ --transcriptome=/pnas/zhangz_group/zhangysh/reference/hg38/cellranger/refdata-cellranger-GRCh38-and-mm10-3.1.0/ \ --localcores=32 --noexit \

/xtdisk/apod-gen/haoll/xiongyj/1-10X/PRJNA577618/1-output/SRX6991953.log 2>&1 &

And I run into an error with "Waiting 6 seconds for UI to do final refresh. Pipestance failed. Use --noexit option to keep UI running after failure." However,when I do put "--noexit", it still not work. Would you please give any suggestion about how to fix it?

############################### Martian Runtime - '3.1.0-v3.2.3' 2020-08-12 15:28:55 [runtime] Reattaching in local mode. Serving UI at http://fat2:37279?auth=BN0YvFY4l8EDcFdkiwphGG0EtiSOWSlPaGbZtVdpMeM

2020-08-12 15:28:58 [runtime] (reset-partial) ID.SRX6991954.SC_RNA_COUNTER_CS.SC_RNA_COUNTER._BASIC_SC_RNA_COUNTER.MARK_DUPLICATES.fork0.chnk04 2020-08-12 15:28:59 [runtime] (reset-partial) ID.SRX6991954.SC_RNA_COUNTER_CS.SC_RNA_COUNTER._BASIC_SC_RNA_COUNTER.MARK_DUPLICATES.fork0.chnk05 2020-08-12 15:28:59 [runtime] (reset-partial) ID.SRX6991954.SC_RNA_COUNTER_CS.SC_RNA_COUNTER._BASIC_SC_RNA_COUNTER.MARK_DUPLICATES.fork0.chnk06 2020-08-12 15:29:00 [runtime] (reset-partial) ID.SRX6991954.SC_RNA_COUNTER_CS.SC_RNA_COUNTER._BASIC_SC_RNA_COUNTER.MARK_DUPLICATES.fork0.chnk07 2020-08-12 15:29:00 [runtime] (reset-partial) ID.SRX6991954.SC_RNA_COUNTER_CS.SC_RNA_COUNTER._BASIC_SC_RNA_COUNTER.MARK_DUPLICATES.fork0.chnk08 2020-08-12 15:29:01 [runtime] (reset-partial) ID.SRX6991954.SC_RNA_COUNTER_CS.SC_RNA_COUNTER._BASIC_SC_RNA_COUNTER.MARK_DUPLICATES.fork0.chnk09 2020-08-12 15:29:01 [runtime] (reset-partial) ID.SRX6991954.SC_RNA_COUNTER_CS.SC_RNA_COUNTER._BASIC_SC_RNA_COUNTER.MARK_DUPLICATES.fork0.chnk10 2020-08-12 15:29:01 [runtime] (reset-partial) ID.SRX6991954.SC_RNA_COUNTER_CS.SC_RNA_COUNTER._BASIC_SC_RNA_COUNTER.MARK_DUPLICATES.fork0.chnk11 2020-08-12 15:29:02 [runtime] Found orphaned local stage: ID.SRX6991954.SC_RNA_COUNTER_CS.SC_RNA_COUNTER._BASIC_SC_RNA_COUNTER.MARK_DUPLICATES 2020-08-12 15:29:02 [runtime] (split_complete) ID.SRX6991954.SC_RNA_COUNTER_CS.SC_RNA_COUNTER._BASIC_SC_RNA_COUNTER.MARK_DUPLICATES 2020-08-12 15:29:02 [runtime] (run:local) ID.SRX6991954.SC_RNA_COUNTER_CS.SC_RNA_COUNTER._BASIC_SC_RNA_COUNTER.MARK_DUPLICATES.fork0.chnk04.main 2020-08-12 15:29:02 [runtime] (run:local) ID.SRX6991954.SC_RNA_COUNTER_CS.SC_RNA_COUNTER._BASIC_SC_RNA_COUNTER.MARK_DUPLICATES.fork0.chnk05.main 2020-08-12 15:29:02 [runtime] (run:local) ID.SRX6991954.SC_RNA_COUNTER_CS.SC_RNA_COUNTER._BASIC_SC_RNA_COUNTER.MARK_DUPLICATES.fork0.chnk06.main 2020-08-12 15:29:02 [runtime] (run:local) ID.SRX6991954.SC_RNA_COUNTER_CS.SC_RNA_COUNTER._BASIC_SC_RNA_COUNTER.MARK_DUPLICATES.fork0.chnk07.main 2020-08-12 15:29:02 [runtime] (run:local) ID.SRX6991954.SC_RNA_COUNTER_CS.SC_RNA_COUNTER._BASIC_SC_RNA_COUNTER.MARK_DUPLICATES.fork0.chnk08.main 2020-08-12 15:29:02 [runtime] (run:local) ID.SRX6991954.SC_RNA_COUNTER_CS.SC_RNA_COUNTER._BASIC_SC_RNA_COUNTER.MARK_DUPLICATES.fork0.chnk09.main 2020-08-12 15:29:02 [runtime] (run:local) ID.SRX6991954.SC_RNA_COUNTER_CS.SC_RNA_COUNTER._BASIC_SC_RNA_COUNTER.MARK_DUPLICATES.fork0.chnk10.main 2020-08-12 15:29:02 [runtime] (run:local) ID.SRX6991954.SC_RNA_COUNTER_CS.SC_RNA_COUNTER._BASIC_SC_RNA_COUNTER.MARK_DUPLICATES.fork0.chnk11.main 2020-08-12 15:30:01 [runtime] (failed) ID.SRX6991954.SC_RNA_COUNTER_CS.SC_RNA_COUNTER._BASIC_SC_RNA_COUNTER.MARK_DUPLICATES

[error] Pipestance failed. Error log at: SRX6991954/SC_RNA_COUNTER_CS/SC_RNA_COUNTER/_BASIC_SC_RNA_COUNTER/MARK_DUPLICATES/fork0/chnk06-ud548339a3b/_errors

Log message: thread 'main' panicked at 'Found 0 or >1 features for confidently mapped read/pair 19627721-GTGCCCGT': cr_stage/src/cmd_mark_dups.rs:285stack backtrace: 0: martian::martian_main::{{closure}}::h91b2cb6e4ec208c2 (0x7f3d3018d444) at /mnt/home/jenkins/workspace/cellranger-3.1.0-2.2.5-build/sake/modules/cellranger-cs/3.1.0/lib/rust/.cargo/git/checkouts/martian-rust-35615836cc90309f/56bdd8d/src/lib.rs:418 1: rust_panic_with_hook (0x7f3d3027f0a8) at src/libstd/panicking.rs:482 2: std::panicking::begin_panic::h57e271f882aa496c (0x7f3d30129a69) at /rustc/fc50f328b0353b285421b8ff5d4100966387a997/src/libstd/panicking.rs:412 3: cr_stage::cmd_mark_dups::process_barcode::hd751fc817f7edb3f (0x7f3d301378b9) at cr_stage/src/cmd_mark_dups.rs:285 4: cr_stage::cmd_mark_dups::cmd_mark_dups::hd092a5adfde3f170 (0x7f3d3013d474) at cr_stage/src/cmd_mark_dups.rs:487

::main::h974808f790ef4388 at cr_stage/src/cmd_mark_dups.rs:571 5: martian::do_main::h2e308a8e2c4f463b (0x7f3d3018bde0) at /mnt/home/jenkins/workspace/cellranger-3.1.0-2.2.5-build/sake/modules/cellranger-cs/3.1.0/lib/rust/.cargo/git/checkouts/martian-rust-35615836cc90309f/56bdd8d/src/lib.rs:320 6: martian::martian_main::h6fd98ef8e453a44e (0x7f3d3018d188) at /mnt/home/jenkins/workspace/cellranger-3.1.0-2.2.5-build/sake/modules/cellranger-cs/3.1.0/lib/rust/.cargo/git/checkouts/martian-rust-35615836cc90309f/56bdd8d/src/lib.rs:456 7: cr_stage::main::hc37a84eb0c22f378 (0x7f3d30159ec6) at cr_stage/src/main.rs:84 8: std::rt::lang_start::{{closure}}::h604fde077e847e29 (0x7f3d3012e3b2) at /rustc/fc50f328b0353b285421b8ff5d4100966387a997/src/libstd/rt.rs:64 9: {{closure}} (0x7f3d3027e9c2) at src/libstd/rt.rs:49 do_call at src/libstd/panicking.rs:297 10: __rust_maybe_catch_panic (0x7f3d30282449) at src/libpanic_unwind/lib.rs:87 11: try (0x7f3d3027f4cc) at src/libstd/panicking.rs:276 catch_unwind at src/libstd/panic.rs:388 lang_start_internal at src/libstd/rt.rs:48 12: main (0x7f3d3015a107) 13: __libc_start_main (0x7f3d2f0d2b34) 14: (0x7f3d3011b2c8) **Waiting 6 seconds for UI to do final refresh. Pipestance failed. Use --noexit option to keep UI running after failure.** 2020-08-12 15:30:07 Shutting down. Saving pipestance info to "SRX6991953/SRX6991953.mri.tgz" For assistance, upload this file to 10x Genomics by running: cellranger upload "SRX6991954/SRX6991954.mri.tgz" 59.22user 2.90system 1:29.84elapsed 69%CPU (0avgtext+0avgdata 100612maxresident)k 208inputs+344576outputs (0major+427698minor)pagefaults 0swaps ######################## However,when I do put "--noexit", it still not work. The command is as followed: nohup time cellranger count --id=SRX6991953 \ --sample=SRX6991953 \ --fastqs=/xtdisk/apod-gen/haoll/xiongyj/1-10X/PRJNA577618/1-output/SRX6991953/ \ --transcriptome=/pnas/zhangz_group/zhangysh/reference/hg38/cellranger/refdata-cellranger-GRCh38-and-mm10-3.1.0/ \ --localcores=32 --noexit \ > /xtdisk/apod-gen/haoll/xiongyj/1-10X/PRJNA577618/1-output/SRX6991953.log 2>&1 & ######################### [error] Pipestance failed. Error log at: SRX6991953/SC_RNA_COUNTER_CS/SC_RNA_COUNTER/_BASIC_SC_RNA_COUNTER/MARK_DUPLICATES/fork0/chnk15-u279b4379a4/_errors Log message: thread 'main' panicked at 'Found 0 or >1 features for confidently mapped read/pair -': cr_stage/src/cmd_mark_dups.rs:285stack backtrace: 0: martian::martian_main::{{closure}}::h91b2cb6e4ec208c2 (0x7f42c4cf3444) at /mnt/home/jenkins/workspace/cellranger-3.1.0-2.2.5-build/sake/modules/cellranger-cs/3.1.0/lib/rust/.cargo/git/checkouts/martian-rust-35615836cc90309f/56bdd8d/src/lib.rs:418 1: rust_panic_with_hook (0x7f42c4de50a8) at src/libstd/panicking.rs:482 2: std::panicking::begin_panic::h57e271f882aa496c (0x7f42c4c8fa69) at /rustc/fc50f328b0353b285421b8ff5d4100966387a997/src/libstd/panicking.rs:412 3: cr_stage::cmd_mark_dups::process_barcode::hd751fc817f7edb3f (0x7f42c4c9d8b9) at cr_stage/src/cmd_mark_dups.rs:285 4: cr_stage::cmd_mark_dups::cmd_mark_dups::hd092a5adfde3f170 (0x7f42c4ca3783) at cr_stage/src/cmd_mark_dups.rs:503 ::main::h974808f790ef4388 at cr_stage/src/cmd_mark_dups.rs:571 5: martian::do_main::h2e308a8e2c4f463b (0x7f42c4cf1de0) at /mnt/home/jenkins/workspace/cellranger-3.1.0-2.2.5-build/sake/modules/cellranger-cs/3.1.0/lib/rust/.cargo/git/checkouts/martian-rust-35615836cc90309f/56bdd8d/src/lib.rs:320 6: martian::martian_main::h6fd98ef8e453a44e (0x7f42c4cf3188) at /mnt/home/jenkins/workspace/cellranger-3.1.0-2.2.5-build/sake/modules/cellranger-cs/3.1.0/lib/rust/.cargo/git/checkouts/martian-rust-35615836cc90309f/56bdd8d/src/lib.rs:456 7: cr_stage::main::hc37a84eb0c22f378 (0x7f42c4cbfec6) at cr_stage/src/main.rs:84 8: std::rt::lang_start::{{closure}}::h604fde077e847e29 (0x7f42c4c943b2) at /rustc/fc50f328b0353b285421b8ff5d4100966387a997/src/libstd/rt.rs:64 9: {{closure}} (0x7f42c4de49c2) at src/libstd/rt.rs:49 do_call at src/libstd/panicking.rs:297 10: __rust_maybe_catch_panic (0x7f42c4de8449) at src/libpanic_unwind/lib.rs:87 11: try (0x7f42c4de54cc) at src/libstd/panicking.rs:276 catch_unwind at src/libstd/panic.rs:388 lang_start_internal at src/libstd/rt.rs:48 12: main (0x7f42c4cc0107) 13: __libc_start_main (0x7f42c3c38b34) 14: (0x7f42c4c812c8) **Pipestance failed, staying alive because --noexit given.**
chuiqin commented 3 years ago

I have the same problem.Did you solve it?

MubasherMohammed commented 3 years ago

Same issue! did you guys managed to solve it?

FEIFEIEIAr commented 3 years ago

Did you solve this? Please help me.

MubasherMohammed commented 3 years ago

It was disc space issue in my case, try to make sure enough space for pipestance pipe line to run

FEIFEIEIAr commented 3 years ago

It was disc space issue in my case, try to make sure enough space for pipestance pipe line to run

How large is your disc?

MubasherMohammed commented 3 years ago

It was disc space issue in my case, try to make sure enough space for pipestance pipe line to run

How large is your disc?

It depends how big is your RNA-seq data to be processed because they’ll be many files to produce by the pipeline. 20G wasn’t enough for me I had to get rid of some stuff

twang15 commented 3 years ago

Any update any this? We see the same issue and have ruled out the possibility of insufficient storage.

    Job failed in stage code
    exit status 255
    Waiting 6 seconds for UI to do final refresh.
    Pipestance failed.
    Use --noexit option to keep UI running after failure.
evolvedmicrobe commented 3 years ago

Hi @twang15 please reach out to support@10xgenomics.com as someone there can help you.