search

pachterlab / kb_python

A wrapper for the kallisto | bustools workflow for single-cell RNA-seq pre-processing
https://www.kallistobus.tools/
BSD 2-Clause "Simplified" License
154 stars 23 forks source link

kb count INDROPSV2 data, the result is not expected #175

Closed monoplasty closed 2 years ago

monoplasty commented 2 years ago

Describe the issue kb count INDROPSV2 data https://data.humancellatlas.org/explore/projects/7c75f07c-608d-4c4a-a1b7-b13d11c0ad31 , Why does so much data generate only a little result? what other input files should I need? Thank you.

Generating whitelist.txt file:

AAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAA
AGTCTCTCAGCGGTTCTGG
CTGATGGCTCAGGAACACG
TCTGATGGCTCGGGAACAC
TTTTTTTTTTTAAAAAAAA
TTTTTTTTTTTTTTTTTTT

What is the exact command that was run?

kb count -i /data/kallisto/refdata/human/transcriptome.idx -g /data/kallisto/refdata/human/transcripts_to_genes.txt -t 16 -m 32G --h5ad --cellranger --verbose  --overwrite  -x INDROPSV2  -o  /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/h5adoutput/ \
/data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/*/*.gz

Command output (with --verbose flag)

[2022-09-27 10:16:13,549]   DEBUG [main] Printing verbose output
[2022-09-27 10:16:15,722]   DEBUG [main] kallisto binary located at /usr/local/python3/lib/python3.9/site-packages/kb_python/bins/linux/kallisto/kallisto
[2022-09-27 10:16:15,723]   DEBUG [main] bustools binary located at /usr/local/python3/lib/python3.9/site-packages/kb_python/bins/linux/bustools/bustools
[2022-09-27 10:16:15,723]   DEBUG [main] Creating `/data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/h5adoutput/tmp` directory
[2022-09-27 10:16:15,723]   DEBUG [main] Namespace(list=False, command='count', tmp=None, keep_tmp=False, verbose=True, i='/data/kallisto/refdata/human/transcriptome.idx', g='/data/kallisto/refdata/human/transcripts_to_genes.txt', x='INDROPSV2', o='/data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/h5adoutput/', w=None, t=16, m='32G', strand=None, workflow='standard', em=False, umi_gene=False, mm=False, tcc=False, filter=None, filter_threshold=None, c1=None, c2=None, overwrite=True, dry_run=False, loom=False, h5ad=True, cellranger=True, gene_names=False, report=False, no_inspect=False, kallisto='/usr/local/python3/lib/python3.9/site-packages/kb_python/bins/linux/kallisto/kallisto', bustools='/usr/local/python3/lib/python3.9/site-packages/kb_python/bins/linux/bustools/bustools', no_validate=False, parity=None, fragment_l=None, fragment_s=None, fastqs=['/data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/0527a1b6-ec32-4b21-bfaf-1be6eaadbdf3/LYMPHNODE2_ATCACG_L005_R1_001.fastq.gz', ...])
[2022-09-27 10:16:18,766]    INFO [count] Using index /data/kallisto/refdata/human/transcriptome.idx to generate BUS file to /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/h5adoutput/ from
[2022-09-27 10:16:18,766]    INFO [count]         /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/0527a1b6-ec32-4b21-bfaf-1be6eaadbdf3/LYMPHNODE2_ATCACG_L005_R1_001.fastq.gz
[2022-09-27 10:16:18,766]    INFO [count]         /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/0527a1b6-ec32-4b21-bfaf-1be6eaadbdf3/LYMPHNODE2_ATCACG_L005_R2_001.fastq.gz
...
[2022-09-27 10:16:18,777]   DEBUG [count] kallisto bus -i /data/kallisto/refdata/human/transcriptome.idx -o /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/h5adoutput/ -x INDROPSV2 -t 16 /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/0527a1b6-ec32-4b21-bfaf-1be6eaadbdf3/LYMPHNODE2_ATCACG_L005_R1_001.fastq.gz 
...
[2022-09-27 10:16:18,888]   DEBUG [count] 
[2022-09-27 10:16:18,888]   DEBUG [count] [bus] Note: Strand option was not specified; setting it to --unstranded for specified technology
[2022-09-27 10:16:18,888]   DEBUG [count] [index] k-mer length: 31
[2022-09-27 10:16:18,888]   DEBUG [count] [index] number of targets: 251,121
[2022-09-27 10:16:18,888]   DEBUG [count] [index] number of k-mers: 149,770,765
[2022-09-27 10:16:47,736]   DEBUG [count] [index] number of equivalence classes: 1,081,681
[2022-09-27 10:16:51,942]   DEBUG [count] [quant] will process sample 1: /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/0527a1b6-ec32-4b21-bfaf-1be6eaadbdf3/LYMPHNODE2_ATCACG_L005_R1_001.fastq.gz
[2022-09-27 10:16:51,943]   DEBUG [count] /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/0527a1b6-ec32-4b21-bfaf-1be6eaadbdf3/LYMPHNODE2_ATCACG_L005_R2_001.fastq.gz
[2022-09-27 10:16:51,943]   DEBUG [count] [quant] will process sample 2: /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/0527a1b6-ec32-4b21-bfaf-1be6eaadbdf3/LYMPHNODE2_ATCACG_L005_RESEQ_R1_001.fastq.gz
[2022-09-27 10:16:51,943]   DEBUG [count] /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/0527a1b6-ec32-4b21-bfaf-1be6eaadbdf3/LYMPHNODE2_ATCACG_L005_RESEQ_R2_001.fastq.gz
[2022-09-27 10:16:51,943]   DEBUG [count] [quant] will process sample 3: /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/08ad15d8-1dfa-4878-86e6-068b12e261d8/JULY_CGC_TUMOR4_ACAGTG_L001_R1_001.fastq.gz
[2022-09-27 10:16:51,943]   DEBUG [count] /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/08ad15d8-1dfa-4878-86e6-068b12e261d8/JULY_CGC_TUMOR4_ACAGTG_L001_R2_001.fastq.gz
[2022-09-27 10:16:51,943]   DEBUG [count] [quant] will process sample 4: /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/08ad15d8-1dfa-4878-86e6-068b12e261d8/TUMOR4_ACAGTG_L001_R1_001.fastq.gz
[2022-09-27 10:16:51,943]   DEBUG [count] /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/08ad15d8-1dfa-4878-86e6-068b12e261d8/TUMOR4_ACAGTG_L001_R2_001.fastq.gz
...
[2022-09-27 10:16:51,952]   DEBUG [count] [quant] will process sample 150: /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/d8e89c86-0da5-4a16-adc8-7801565b1e9a/JULY_BLOOD1_CGATGT_L006_R1_001.fastq.gz
[2022-09-27 10:16:51,952]   DEBUG [count] /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/d8e89c86-0da5-4a16-adc8-7801565b1e9a/JULY_BLOOD1_CGATGT_L006_R2_001.fastq.gz
[2022-09-27 10:16:51,952]   DEBUG [count] [quant] will process sample 151: /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/d8e89c86-0da5-4a16-adc8-7801565b1e9a/JULY_BLOOD1_CGATGT_L007_R1_001.fastq.gz
[2022-09-27 10:16:51,952]   DEBUG [count] /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/d8e89c86-0da5-4a16-adc8-7801565b1e9a/JULY_BLOOD1_CGATGT_L007_R2_001.fastq.gz
[2022-09-27 10:16:51,952]   DEBUG [count] [quant] will process sample 152: /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/d8ed484b-36eb-4be1-bdc6-1f50690a9b56/JULY_TUMOR5_CGATGT_L007_R1_001.fastq.gz
[2022-09-27 10:16:51,952]   DEBUG [count] /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/d8ed484b-36eb-4be1-bdc6-1f50690a9b56/JULY_TUMOR5_CGATGT_L007_R1_002.fastq.gz
[2022-09-27 10:16:51,952]   DEBUG [count] [quant] will process sample 153: /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/d8ed484b-36eb-4be1-bdc6-1f50690a9b56/JULY_TUMOR5_CGATGT_L007_R2_001.fastq.gz
[2022-09-27 10:16:51,952]   DEBUG [count] /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/d8ed484b-36eb-4be1-bdc6-1f50690a9b56/JULY_TUMOR5_CGATGT_L007_R2_002.fastq.gz
[2022-09-27 10:16:51,952]   DEBUG [count] [quant] will process sample 154: /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/d8ed484b-36eb-4be1-bdc6-1f50690a9b56/TUMOR5_CGATGT_L003_R1_001.fastq.gz
[2022-09-27 10:16:51,952]   DEBUG [count] /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/d8ed484b-36eb-4be1-bdc6-1f50690a9b56/TUMOR5_CGATGT_L003_R1_002.fastq.gz
[2022-09-27 10:16:51,952]   DEBUG [count] [quant] will process sample 155: /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/d8ed484b-36eb-4be1-bdc6-1f50690a9b56/TUMOR5_CGATGT_L003_R2_001.fastq.gz
[2022-09-27 10:16:51,952]   DEBUG [count] /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/d8ed484b-36eb-4be1-bdc6-1f50690a9b56/TUMOR5_CGATGT_L003_R2_002.fastq.gz
[2022-09-27 10:16:51,952]   DEBUG [count] [quant] will process sample 156: /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/d8f0e6a4-6ce6-4c6c-b030-72c36e5b33d5/BLOOD4_TTAGGC_L005_R1_001.fastq.gz
[2022-09-27 10:16:51,952]   DEBUG [count] /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/d8f0e6a4-6ce6-4c6c-b030-72c36e5b33d5/BLOOD4_TTAGGC_L005_R2_001.fastq.gz
[2022-09-27 10:16:51,952]   DEBUG [count] [quant] will process sample 157: /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/d8f0e6a4-6ce6-4c6c-b030-72c36e5b33d5/JULY_BLOOD4_TTAGGC_L001_R1_001.fastq.gz
[2022-09-27 10:16:51,953]   DEBUG [count] /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/d8f0e6a4-6ce6-4c6c-b030-72c36e5b33d5/JULY_BLOOD4_TTAGGC_L001_R2_001.fastq.gz
[2022-09-27 10:16:51,953]   DEBUG [count] [quant] will process sample 158: /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/da81289e-aa46-44c9-b429-01c04613ae53/JULY_CGC_NORMAL1_ATCACG_L003_R1_001.fastq.gz
[2022-09-27 10:16:51,953]   DEBUG [count] /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/da81289e-aa46-44c9-b429-01c04613ae53/JULY_CGC_NORMAL1_ATCACG_L003_R2_001.fastq.gz
[2022-09-27 10:16:51,953]   DEBUG [count] [quant] will process sample 159: /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/da81289e-aa46-44c9-b429-01c04613ae53/NORMAL1_ATCACG_L003_R1_001.fastq.gz
[2022-09-27 10:16:51,953]   DEBUG [count] /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/da81289e-aa46-44c9-b429-01c04613ae53/NORMAL1_ATCACG_L003_R2_001.fastq.gz
[2022-09-27 10:16:51,953]   DEBUG [count] [quant] will process sample 160: /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/dac1c1f9-dabe-4d63-9ef1-0e9672673007/DJ010_NoIndex_L008_R1_001.fastq.gz
[2022-09-27 10:16:51,953]   DEBUG [count] /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/dac1c1f9-dabe-4d63-9ef1-0e9672673007/DJ010_NoIndex_L008_R1_002.fastq.gz
[2022-09-27 10:16:51,953]   DEBUG [count] [quant] will process sample 161: /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/dac1c1f9-dabe-4d63-9ef1-0e9672673007/DJ010_NoIndex_L008_R1_003.fastq.gz
[2022-09-27 10:16:51,953]   DEBUG [count] /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/dac1c1f9-dabe-4d63-9ef1-0e9672673007/DJ010_NoIndex_L008_R3_001.fastq.gz
[2022-09-27 10:16:51,953]   DEBUG [count] [quant] will process sample 162: /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/dac1c1f9-dabe-4d63-9ef1-0e9672673007/DJ010_NoIndex_L008_R3_002.fastq.gz
[2022-09-27 10:16:51,953]   DEBUG [count] /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/dac1c1f9-dabe-4d63-9ef1-0e9672673007/DJ010_NoIndex_L008_R3_003.fastq.gz
[2022-09-27 10:16:51,953]   DEBUG [count] [quant] will process sample 163: /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/db523266-4179-4dc0-ba3e-f41e9c7c6448/DJ023_NoIndex_L004_R1_001.fastq.gz
[2022-09-27 10:16:51,953]   DEBUG [count] /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/db523266-4179-4dc0-ba3e-f41e9c7c6448/DJ023_NoIndex_L004_R1_002.fastq.gz
[2022-09-27 10:16:51,953]   DEBUG [count] [quant] will process sample 164: /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/db523266-4179-4dc0-ba3e-f41e9c7c6448/DJ023_NoIndex_L004_R1_003.fastq.gz
[2022-09-27 10:16:51,953]   DEBUG [count] /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/db523266-4179-4dc0-ba3e-f41e9c7c6448/DJ023_NoIndex_L004_R1_004.fastq.gz
[2022-09-27 10:16:51,953]   DEBUG [count] [quant] will process sample 165: /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/db523266-4179-4dc0-ba3e-f41e9c7c6448/DJ023_NoIndex_L004_R3_001.fastq.gz
[2022-09-27 10:16:51,953]   DEBUG [count] /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/db523266-4179-4dc0-ba3e-f41e9c7c6448/DJ023_NoIndex_L004_R3_002.fastq.gz
[2022-09-27 10:16:51,953]   DEBUG [count] [quant] will process sample 166: /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/db523266-4179-4dc0-ba3e-f41e9c7c6448/DJ023_NoIndex_L004_R3_003.fastq.gz
[2022-09-27 10:16:51,953]   DEBUG [count] /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/db523266-4179-4dc0-ba3e-f41e9c7c6448/DJ023_NoIndex_L004_R3_004.fastq.gz
[2022-09-27 10:16:51,953]   DEBUG [count] [quant] will process sample 167: /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/e4ca8c56-c33b-4f40-bf73-f2948d1c87fd/BC11_P4_1_TCR_IGO_08295_D_1_S1_L001_R1_001.fastq.gz
[2022-09-27 10:16:51,953]   DEBUG [count] /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/e4ca8c56-c33b-4f40-bf73-f2948d1c87fd/BC11_P4_1_TCR_IGO_08295_D_1_S1_L001_R2_001.fastq.gz
[2022-09-27 10:16:51,953]   DEBUG [count] [quant] will process sample 168: /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/e4ca8c56-c33b-4f40-bf73-f2948d1c87fd/BC11_P4_1_TCR_IGO_08295_D_1_S1_L002_R1_001.fastq.gz
[2022-09-27 10:16:51,953]   DEBUG [count] /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/e4ca8c56-c33b-4f40-bf73-f2948d1c87fd/BC11_P4_1_TCR_IGO_08295_D_1_S1_L002_R2_001.fastq.gz
[2022-09-27 10:16:51,953]   DEBUG [count] [quant] will process sample 169: /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/e740aa37-dbd8-4a29-8f03-45cc26c953ea/BLOOD5_TGACCA_L006_R1_001.fastq.gz
[2022-09-27 10:16:51,953]   DEBUG [count] /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/e740aa37-dbd8-4a29-8f03-45cc26c953ea/BLOOD5_TGACCA_L006_R1_002.fastq.gz
[2022-09-27 10:16:51,953]   DEBUG [count] [quant] will process sample 170: /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/e740aa37-dbd8-4a29-8f03-45cc26c953ea/BLOOD5_TGACCA_L006_R2_001.fastq.gz
[2022-09-27 10:16:51,953]   DEBUG [count] /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/e740aa37-dbd8-4a29-8f03-45cc26c953ea/BLOOD5_TGACCA_L006_R2_002.fastq.gz
[2022-09-27 10:16:51,953]   DEBUG [count] [quant] will process sample 171: /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/e740aa37-dbd8-4a29-8f03-45cc26c953ea/JULY_BLOOD5_TGACCA_L002_R1_001.fastq.gz
[2022-09-27 10:16:51,953]   DEBUG [count] /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/e740aa37-dbd8-4a29-8f03-45cc26c953ea/JULY_BLOOD5_TGACCA_L002_R2_001.fastq.gz
[2022-09-27 10:16:51,953]   DEBUG [count] [quant] will process sample 172: /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/e91394b8-205c-432c-8cd7-982db378d282/NORMAL1_CGATGT_L008_R1_001.fastq.gz
[2022-09-27 10:16:51,953]   DEBUG [count] /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/e91394b8-205c-432c-8cd7-982db378d282/NORMAL1_CGATGT_L008_R2_001.fastq.gz
[2022-09-27 10:16:51,953]   DEBUG [count] [quant] will process sample 173: /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/e91394b8-205c-432c-8cd7-982db378d282/Normal_1_IGO_06811_4_S3_L003_R1_001.fastq.gz
[2022-09-27 10:16:51,954]   DEBUG [count] /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/e91394b8-205c-432c-8cd7-982db378d282/Normal_1_IGO_06811_4_S3_L003_R2_001.fastq.gz
[2022-09-27 10:16:51,954]   DEBUG [count] [quant] will process sample 174: /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/e927a92d-d838-475e-b15a-d52284b3ed02/BC01_blood_1_IGO_06811_1_S1_L001_R1_001.fastq.gz
[2022-09-27 10:16:51,954]   DEBUG [count] /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/e927a92d-d838-475e-b15a-d52284b3ed02/BC01_blood_1_IGO_06811_1_S1_L001_R2_001.fastq.gz
[2022-09-27 10:16:51,954]   DEBUG [count] [quant] will process sample 175: /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/ed42bf4e-234b-42c8-a6a0-959715959d52/JULY_TUMOR2_TGACCA_L005_R1_001.fastq.gz
[2022-09-27 10:16:51,954]   DEBUG [count] /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/ed42bf4e-234b-42c8-a6a0-959715959d52/JULY_TUMOR2_TGACCA_L005_R1_002.fastq.gz
[2022-09-27 10:16:51,954]   DEBUG [count] [quant] will process sample 176: /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/ed42bf4e-234b-42c8-a6a0-959715959d52/JULY_TUMOR2_TGACCA_L005_R2_001.fastq.gz
[2022-09-27 10:16:51,954]   DEBUG [count] /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/ed42bf4e-234b-42c8-a6a0-959715959d52/JULY_TUMOR2_TGACCA_L005_R2_002.fastq.gz
[2022-09-27 10:16:51,954]   DEBUG [count] [quant] will process sample 177: /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/ed42bf4e-234b-42c8-a6a0-959715959d52/TUMOR2_TGACCA_L001_R1_002.fastq.gz
[2022-09-27 10:16:51,954]   DEBUG [count] /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/ed42bf4e-234b-42c8-a6a0-959715959d52/TUMOR2_TGACCA_L001_R2_002.fastq.gz
[2022-09-27 10:16:51,954]   DEBUG [count] [quant] will process sample 178: /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/f6c5ed9c-995f-4096-a803-611d7f628ab6/DJ020_NoIndex_L001_R1_001.fastq.gz
[2022-09-27 10:16:51,954]   DEBUG [count] /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/f6c5ed9c-995f-4096-a803-611d7f628ab6/DJ020_NoIndex_L001_R1_002.fastq.gz
[2022-09-27 10:16:51,954]   DEBUG [count] [quant] will process sample 179: /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/f6c5ed9c-995f-4096-a803-611d7f628ab6/DJ020_NoIndex_L001_R1_003.fastq.gz
[2022-09-27 10:16:51,954]   DEBUG [count] /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/f6c5ed9c-995f-4096-a803-611d7f628ab6/DJ020_NoIndex_L001_R3_001.fastq.gz
[2022-09-27 10:16:51,954]   DEBUG [count] [quant] will process sample 180: /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/f6c5ed9c-995f-4096-a803-611d7f628ab6/DJ020_NoIndex_L001_R3_002.fastq.gz
[2022-09-27 10:16:51,954]   DEBUG [count] /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/f6c5ed9c-995f-4096-a803-611d7f628ab6/DJ020_NoIndex_L001_R3_003.fastq.gz
[2022-09-27 10:16:51,954]   DEBUG [count] [quant] will process sample 181: /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/fd7d0af4-2ec0-457c-bde8-de95f1fc920b/DJ008_NoIndex_L007_R1_001.fastq.gz
[2022-09-27 10:16:51,954]   DEBUG [count] /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/fd7d0af4-2ec0-457c-bde8-de95f1fc920b/DJ008_NoIndex_L007_R1_002.fastq.gz
[2022-09-27 10:16:51,954]   DEBUG [count] [quant] will process sample 182: /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/fd7d0af4-2ec0-457c-bde8-de95f1fc920b/DJ008_NoIndex_L007_R1_003.fastq.gz
[2022-09-27 10:16:51,954]   DEBUG [count] /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/fd7d0af4-2ec0-457c-bde8-de95f1fc920b/DJ008_NoIndex_L007_R3_001.fastq.gz
[2022-09-27 10:16:51,954]   DEBUG [count] [quant] will process sample 183: /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/fd7d0af4-2ec0-457c-bde8-de95f1fc920b/DJ008_NoIndex_L007_R3_002.fastq.gz
[2022-09-27 10:16:51,954]   DEBUG [count] /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/fd7d0af4-2ec0-457c-bde8-de95f1fc920b/DJ008_NoIndex_L007_R3_003.fastq.gz
[2022-09-27 10:16:51,954]   DEBUG [count] [quant] will process sample 184: /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/ff042dab-a560-407d-9c7e-91058ce633a6/JULY_CGC_TUMOR2_TGACCA_L002_R1_001.fastq.gz
[2022-09-27 10:16:51,954]   DEBUG [count] /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/ff042dab-a560-407d-9c7e-91058ce633a6/JULY_CGC_TUMOR2_TGACCA_L002_R2_001.fastq.gz
[2022-09-27 10:16:51,954]   DEBUG [count] [quant] will process sample 185: /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/ff042dab-a560-407d-9c7e-91058ce633a6/TUMOR2_TGACCA_L002_R1_001.fastq.gz
[2022-09-27 10:16:51,954]   DEBUG [count] /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/ff042dab-a560-407d-9c7e-91058ce633a6/TUMOR2_TGACCA_L002_R2_001.fastq.gz
[2022-09-27 11:54:20,142]   DEBUG [count] [quant] finding pseudoalignments for the reads ... done
[2022-09-27 11:54:20,163]   DEBUG [count] [quant] processed 8,735,887,445 reads, 1,423,892,584 reads pseudoaligned
[2022-09-27 11:54:21,165]   DEBUG [count] 
[2022-09-27 12:01:29,239]   DEBUG [count] /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/h5adoutput/output.bus passed validation
[2022-09-27 12:01:29,248]    INFO [count] Sorting BUS file /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/h5adoutput/output.bus to /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/h5adoutput/tmp/output.s.bus
[2022-09-27 12:01:29,248]   DEBUG [count] bustools sort -o /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/h5adoutput/tmp/output.s.bus -T /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/h5adoutput/tmp -t 16 -m 32G /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/h5adoutput/output.bus
[2022-09-27 12:13:05,798]   DEBUG [count] Read in 1423892584 BUS records
[2022-09-27 12:16:57,331]   DEBUG [count] /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/h5adoutput/tmp/output.s.bus passed validation
[2022-09-27 12:16:57,339]    INFO [count] Whitelist not provided
[2022-09-27 12:16:57,465]    INFO [count] Generating whitelist /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/h5adoutput/whitelist.txt from BUS file /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/h5adoutput/tmp/output.s.bus
[2022-09-27 12:16:57,465]   DEBUG [count] bustools whitelist -o /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/h5adoutput/whitelist.txt /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/h5adoutput/tmp/output.s.bus
[2022-09-27 12:17:05,383]   DEBUG [count] Read in 752549898 BUS records, wrote 15 barcodes to whitelist with threshold 4508166
[2022-09-27 12:17:05,398]    INFO [count] Inspecting BUS file /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/h5adoutput/tmp/output.s.bus
[2022-09-27 12:17:05,398]   DEBUG [count] bustools inspect -o /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/h5adoutput/inspect.json -w /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/h5adoutput/whitelist.txt /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/h5adoutput/tmp/output.s.bus
[2022-09-27 12:17:38,671]    INFO [count] Correcting BUS records in /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/h5adoutput/tmp/output.s.bus to /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/h5adoutput/tmp/output.s.c.bus with whitelist /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/h5adoutput/whitelist.txt
[2022-09-27 12:17:38,674]   DEBUG [count] bustools correct -o /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/h5adoutput/tmp/output.s.c.bus -w /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/h5adoutput/whitelist.txt /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/h5adoutput/tmp/output.s.bus
[2022-09-27 12:17:38,779]   DEBUG [count] Found 6 barcodes in the whitelist
[2022-09-27 12:18:12,533]   DEBUG [count] Processed 752549898 BUS records
[2022-09-27 12:18:12,533]   DEBUG [count] In whitelist = 2584642
[2022-09-27 12:18:12,533]   DEBUG [count] Corrected    = 3299034
[2022-09-27 12:18:12,533]   DEBUG [count] Uncorrected  = 746666222
[2022-09-27 12:18:15,043]   DEBUG [count] /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/h5adoutput/tmp/output.s.c.bus passed validation
[2022-09-27 12:18:15,071]    INFO [count] Sorting BUS file /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/h5adoutput/tmp/output.s.c.bus to /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/h5adoutput/output.unfiltered.bus
[2022-09-27 12:18:15,071]   DEBUG [count] bustools sort -o /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/h5adoutput/output.unfiltered.bus -T /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/h5adoutput/tmp -t 16 -m 32G /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/h5adoutput/tmp/output.s.c.bus
[2022-09-27 12:18:38,019]   DEBUG [count] Read in 5883676 BUS records
[2022-09-27 12:18:42,943]   DEBUG [count] /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/h5adoutput/output.unfiltered.bus passed validation
[2022-09-27 12:18:43,020]    INFO [count] Generating count matrix /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/h5adoutput/counts_unfiltered/cells_x_genes from BUS file /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/h5adoutput/output.unfiltered.bus
[2022-09-27 12:18:43,070]   DEBUG [count] bustools count -o /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/h5adoutput/counts_unfiltered/cells_x_genes -g /data/kallisto/refdata/human/transcripts_to_genes.txt -e /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/h5adoutput/matrix.ec -t /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/h5adoutput/transcripts.txt --genecounts /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/h5adoutput/output.unfiltered.bus
[2022-09-27 12:18:49,357]   DEBUG [count] /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/h5adoutput/counts_unfiltered/cells_x_genes.mtx passed validation
[2022-09-27 12:18:49,385]    INFO [count] Reading matrix /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/h5adoutput/counts_unfiltered/cells_x_genes.mtx
[2022-09-27 12:19:10,301] WARNING [count] 20453 gene IDs do not have corresponding gene names. These genes will use their gene IDs instead.
[2022-09-27 12:19:10,326]    INFO [count] Writing matrix to h5ad /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/h5adoutput/counts_unfiltered/adata.h5ad
[2022-09-27 12:19:10,779]    INFO [count] Writing matrix in cellranger format to /data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/h5adoutput/counts_unfiltered/cellranger
[2022-09-27 12:19:11,028]   DEBUG [main] Removing `/data/kallisto/fastqs/GSE114727_BreastTumorMicroenvironment/indropsv2/h5adoutput/tmp` directory
github-actions[bot] commented 2 years ago

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days