search

STOmics / SAW

GNU General Public License v3.0
143 stars 34 forks source link

SAW 6.1 ImageQC 2.1 tar package name issue #66

Closed OnkarMulay closed 1 year ago

OnkarMulay commented 1 year ago 
[INFO 20230818-05-26-49 p650958 prepare_input main.py:548] Extract files from zipped file completed.
I0818 05:26:49.319389 650958 main.py:548] Extract files from zipped file completed.
[INFO 20230818-05-26-49 p650958 pipeline_entry main.py:93] Copy ipr.
I0818 05:26:49.321285 650958 main.py:93] Copy ipr.
[2023-08-18 05:26:49] SAW-A40002:    The tar package name is inconsistent with the QCResultFile name.
[2023-08-18 05:26:49] SAW-A40002:    The tar package name is inconsistent with the QCResultFile name.
E0818 05:26:49.326358 650958 main.py:162] SAW-A40002:    The tar package name is inconsistent with the QCResultFile name.
Traceback (most recent call last):
  File "register/stereo-resep-cell_v3.3.4/stereo-resep/main.py", line 879, in <module>
  File "register/stereo-resep-cell_v3.3.4/stereo-resep/main.py", line 37, in __init__
  File "register/stereo-resep-cell_v3.3.4/stereo-resep/main.py", line 122, in pipeline_entry
  File "register/stereo-resep-cell_v3.3.4/stereo-resep/main.py", line 163, in run
Exception: SAW-A40002:    The tar package name is inconsistent with the QCResultFile name
Command exited with non-zero status 1
        Command being timed: "singularity exec /scratch/project/stseq/Onkar/STOmics/stomics_cell_segment/SAW6_1/SAW_6.1.sif register -i /scratch/project/stseq/Onkar/STOmics/stomics_cell_segment/brain/Y1/stich_corrected_Y1_FP200000571BR_D5/FP200000571BR_D5_20211115.tar.gz -c /scratch/project/stseq/Onkar/STOmics/stomics_cell_segment/brain/Y1/stich_corrected_Y1_FP200000571BR_D5/FP200000571BR_D5_SC_20230807_160227_2.1.0.ipr -v /scratch/project/stseq/Onkar/STOmics/stomics_cell_segment/Outputs/brain_Y1//02.count/FP200000571BR_D5.raw.gef -o /scratch/project/stseq/Onkar/STOmics/stomics_cell_segment/Outputs/brain_Y1//03.register"

--- Complete error log

        Command being timed: "singularity exec /scratch/project/stseq/Onkar/STOmics/stomics_cell_segment/SAW6_1/SAW_6.1.sif CIDCount -i /scratch/project/stseq/Onkar/STOmics/stomics_cell_segment/brain/Y1/01.Filter/FP200000571BR_D5.barcodeToPos.h5 -s mouse -g 5"
        User time (seconds): 19.72
        System time (seconds): 0.71
        Percent of CPU this job got: 108%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 0:18.81
        Average shared text size (kbytes): 0
        Average unshared data size (kbytes): 0
        Average stack size (kbytes): 0
        Average total size (kbytes): 0
        Maximum resident set size (kbytes): 708916
        Average resident set size (kbytes): 0
        Major (requiring I/O) page faults: 144
        Minor (reclaiming a frame) page faults: 115449
        Voluntary context switches: 265254
        Involuntary context switches: 30
        Swaps: 0
        File system inputs: 21280
        File system outputs: 0
        Socket messages sent: 0
        Socket messages received: 0
        Signals delivered: 0
        Page size (bytes): 4096
        Exit status: 0
WARNING: SINGULARITY_BIND and APPTAINER_BIND have different values, using the latter
        Command being timed: "singularity exec /scratch/project/stseq/Onkar/STOmics/stomics_cell_segment/SAW6_1/SAW_6.1.sif mapping --outSAMattributes spatial --outSAMtype BAM SortedByCoordinate --genomeDir /scratch/project/stseq/Onkar/STOmics/stomics_cell_segment/NEW_STAR_SJ100 --runThreadN 120 --outFileNamePrefix /scratch/project/stseq/Onkar/STOmics/stomics_cell_segment/Outputs/brain_Y1//00.mapping/DP8400025874TR_L01_read_1. --sysShell /bin/bash --stParaFile /scratch/project/stseq/Onkar/STOmics/stomics_cell_segment/Outputs/brain_Y1//00.mapping/DP8400025874TR_L01_read_1.bcPara --readNameSeparator " " --limitBAMsortRAM 63168332971 --limitOutSJcollapsed 10000000 --limitIObufferSize=280000000 --outBAMsortingBinsN 50"
        User time (seconds): 109016.91
        System time (seconds): 1029.54
        Percent of CPU this job got: 1305%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 2:20:28
        Average shared text size (kbytes): 0
        Average unshared data size (kbytes): 0
        Average stack size (kbytes): 0
        Average total size (kbytes): 0
        Maximum resident set size (kbytes): 170819176
        Average resident set size (kbytes): 0
        Major (requiring I/O) page faults: 171
        Minor (reclaiming a frame) page faults: 23646553
        Voluntary context switches: 5362923
        Involuntary context switches: 254421
        Swaps: 0
        File system inputs: 26240
        File system outputs: 0
        Socket messages sent: 0
        Socket messages received: 0
        Signals delivered: 0
        Page size (bytes): 4096
        Exit status: 0
WARNING: SINGULARITY_BIND and APPTAINER_BIND have different values, using the latter
        Command being timed: "singularity exec /scratch/project/stseq/Onkar/STOmics/stomics_cell_segment/SAW6_1/SAW_6.1.sif count -i /scratch/project/stseq/Onkar/STOmics/stomics_cell_segment/Outputs/brain_Y1//00.mapping/DP8400025874TR_L01_read_1.Aligned.sortedByCoord.out.bam -o /scratch/project/stseq/Onkar/STOmics/stomics_cell_segment/Outputs/brain_Y1//02.count/FP200000571BR_D5.Aligned.sortedByCoord.out.merge.q10.dedup.target.bam -a /scratch/project/stseq/Onkar/STOmics/stomics_cell_segment/Mouse_reference_v5/mouse/genes/Mus_musculus.GRCm39.107.gtf -s /scratch/project/stseq/Onkar/STOmics/stomics_cell_segment/Outputs/brain_Y1//02.count/FP200000571BR_D5.Aligned.sortedByCoord.out.merge.q10.dedup.target.bam.summary.stat -e /scratch/project/stseq/Onkar/STOmics/stomics_cell_segment/Outputs/brain_Y1//02.count/FP200000571BR_D5.raw.gef --sat_file /scratch/project/stseq/Onkar/STOmics/stomics_cell_segment/Outputs/brain_Y1//02.count/FP200000571BR_D5_raw_barcode_gene_exp.txt --umi_on --umi_len 10 --save_lq --save_dup --sn FP200000571BR_D5 -c 120 -m 128"
        User time (seconds): 20792.96
        System time (seconds): 991.13
        Percent of CPU this job got: 1604%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 22:37.35
        Average shared text size (kbytes): 0
        Average unshared data size (kbytes): 0
        Average stack size (kbytes): 0
        Average total size (kbytes): 0
        Maximum resident set size (kbytes): 132177840
        Average resident set size (kbytes): 0
        Major (requiring I/O) page faults: 247
        Minor (reclaiming a frame) page faults: 145139151
        Voluntary context switches: 50596121
        Involuntary context switches: 369673
        Swaps: 0
        File system inputs: 37040
        File system outputs: 0
        Socket messages sent: 0
        Socket messages received: 0
        Signals delivered: 0
        Page size (bytes): 4096
        Exit status: 0
WARNING: SINGULARITY_BIND and APPTAINER_BIND have different values, using the latter
I0818 05:26:32.376527 650958 utils.py:142] Note: detected 192 virtual cores but NumExpr set to maximum of 64, check "NUMEXPR_MAX_THREADS" environment variable.
I0818 05:26:32.376642 650958 utils.py:145] Note: NumExpr detected 192 cores but "NUMEXPR_MAX_THREADS" not set, so enforcing safe limit of 8.
I0818 05:26:32.376688 650958 utils.py:157] NumExpr defaulting to 8 threads.
I0818 05:26:36.140510 650958 schemas.py:1221] Resource 'XMLSchema.xsd' is already loaded
2023-08-18 05:26:41.209488: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /home/s4634945/.local/lib/python3.8/site-packages/cv2/../../lib64:/opt/lib:/.singularity.d/libs
2023-08-18 05:26:41.209584: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
[INFO 20230818-05-26-48 p650958 prepare_dir main.py:473] Dir preparation completed.
I0818 05:26:48.736808 650958 main.py:473] Dir preparation completed.
[WARN 20230818-05-26-48 p650958 check_files2 main.py:497] The *_transformed.tif image could not be found under the /scratch/project/stseq/Onkar/STOmics/stomics_cell_segment/brain/Y1/stich_corrected_Y1_FP200000571BR_D5/FP200000571BR_D5_20211115.tar.gz.
W0818 05:26:48.737285 650958 main.py:497] The *_transformed.tif image could not be found under the /scratch/project/stseq/Onkar/STOmics/stomics_cell_segment/brain/Y1/stich_corrected_Y1_FP200000571BR_D5/FP200000571BR_D5_20211115.tar.gz.
[INFO 20230818-05-26-49 p650958 prepare_input main.py:548] Extract files from zipped file completed.
I0818 05:26:49.319389 650958 main.py:548] Extract files from zipped file completed.
[INFO 20230818-05-26-49 p650958 pipeline_entry main.py:93] Copy ipr.
I0818 05:26:49.321285 650958 main.py:93] Copy ipr.
[2023-08-18 05:26:49] SAW-A40002:    The tar package name is inconsistent with the QCResultFile name.
[2023-08-18 05:26:49] SAW-A40002:    The tar package name is inconsistent with the QCResultFile name.
E0818 05:26:49.326358 650958 main.py:162] SAW-A40002:    The tar package name is inconsistent with the QCResultFile name.
Traceback (most recent call last):
  File "register/stereo-resep-cell_v3.3.4/stereo-resep/main.py", line 879, in <module>
  File "register/stereo-resep-cell_v3.3.4/stereo-resep/main.py", line 37, in __init__
  File "register/stereo-resep-cell_v3.3.4/stereo-resep/main.py", line 122, in pipeline_entry
  File "register/stereo-resep-cell_v3.3.4/stereo-resep/main.py", line 163, in run
Exception: SAW-A40002:    The tar package name is inconsistent with the QCResultFile name
Command exited with non-zero status 1
        Command being timed: "singularity exec /scratch/project/stseq/Onkar/STOmics/stomics_cell_segment/SAW6_1/SAW_6.1.sif register -i /scratch/project/stseq/Onkar/STOmics/stomics_cell_segment/brain/Y1/stich_corrected_Y1_FP200000571BR_D5/FP200000571BR_D5_20211115.tar.gz -c /scratch/project/stseq/Onkar/STOmics/stomics_cell_segment/brain/Y1/stich_corrected_Y1_FP200000571BR_D5/FP200000571BR_D5_SC_20230807_160227_2.1.0.ipr -v /scratch/project/stseq/Onkar/STOmics/stomics_cell_segment/Outputs/brain_Y1//02.count/FP200000571BR_D5.raw.gef -o /scratch/project/stseq/Onkar/STOmics/stomics_cell_segment/Outputs/brain_Y1//03.register"
        User time (seconds): 9.11
        System time (seconds): 3.62
        Percent of CPU this job got: 62%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 0:20.40
        Average shared text size (kbytes): 0
        Average unshared data size (kbytes): 0
        Average stack size (kbytes): 0
        Average total size (kbytes): 0
        Maximum resident set size (kbytes): 768208
        Average resident set size (kbytes): 0
        Major (requiring I/O) page faults: 4502
        Minor (reclaiming a frame) page faults: 185459
        Voluntary context switches: 72033
        Involuntary context switches: 563091
        Swaps: 0
        File system inputs: 863520
        File system outputs: 0
        Socket messages sent: 0
        Socket messages received: 0
        Signals delivered: 0
        Page size (bytes): 4096
        Exit status: 1

Can I get some help in knowing how the name of the tar.gz and .ipr file should be?

For a file that was successful, this is the name:

FP200000571BR_D4_SC_20230720230725_173812_stch.ipr FP200000571BR_D4_SC_20230724_144309_2.0.2.tar.gz

For a file that was unsuccessful, this is the name:

FP200000571BR_E2_SC_20230808_170512_2.1.0.ipr FP200000571BR_E2_20211115.tar.gz

I renamed it to:- FP200000571BR_E2_SC_20230807_170512_2.1.0.tar.gz

But still the same error

Regards, Onkar

TheSallyGardens commented 1 year ago

I suggest that you use imageStudio again for QC on the original images in the tar.gz package.

Best.

OnkarMulay commented 1 year ago

Ok, sure. Thanks. I'll re-run and update.