marcelm
commented
3 years ago Thanks for providing a full log file; that makes it easier to find out what is going on!
The problem appears to be the primer trimming step in IgDiscover. IgDiscover uses Cutadapt to remove them and it also uses the --discard-untrimmed option which makes it throw away all reads that do not contain a primer. Since you have already removed primer sequences, all reads are thrown away. You can see that in the log file:
Total reads processed: 281,290
Reads with adapters: 0 (0.0%)
Reads written (passing filters): 0 (0.0%)Creating the stats/trimming-succesful file by hand unfortunately cannot solve the problem because IgDiscover will then just try to continue with an empty file.
IgDiscover should work correctly if you just give it the unprocessed reads. It will then merge paired-end reads and remove primers. If you want to continue to do these steps yourself, you need to not provide any primers. That is, give an empty list of primers in for the forward_primers: and reverse_primers: settings in the configuration.
bridgelan
It seems that "igdiscover dereplicate" is not processing any sequence. I installed IgDiscover through Conda in a Ubuntu 18.04 LTS. The log printed on Terminal and my configuration file are attached here.
igdiscover.yaml.txt Terminal_IgDiscoverError.txt
The error seems to be here:
`igdiscover dereplicate --json=stats/groups.json --minimum-length=300 reads/4-trimmed.fasta.gz | pigz > reads/sequences.fasta.gz INFO: 0 sequences processed INFO: 0 sequences long enough INFO: 0 dereplicated sequences written INFO: CPU time 00:00:01.3. Maximum memory usage 0.200 GB [Sun Jul 4 21:17:48 2021] Finished job 45. 1 of 44 steps (2%) done
[Sun Jul 4 21:17:48 2021] rule igdiscover_igblast: input: reads/sequences.fasta.gz, iteration-01/database/V.fasta, iteration-01/database/D.fasta, iteration-01/database/J.fasta output: iteration-01/assigned.tab.gz, iteration-01/stats/assigned.json jobid: 52 wildcards: dir=iteration-01 threads: 8
time igdiscover igblast --sequence-type=Ig --rename 'testV3D0M'_ --threads=8 --stats=iteration-01/stats/assigned.json iteration-01/database reads/sequences.fasta.gz | pigz > iteration-01/assigned.tab.gz INFO: Running IgBLAST on database sequences to find CDR/FR region locations INFO: Running IgBLAST on input reads Traceback (most recent call last): File "/home/bridgelan/miniconda3/envs/igdiscover/bin/igdiscover", line 10, in
sys.exit(main())
File "/home/bridgelan/miniconda3/envs/igdiscover/lib/python3.8/site-packages/igdiscover/main.py", line 95, in main
to_run()
File "/home/bridgelan/miniconda3/envs/igdiscover/lib/python3.8/site-packages/igdiscover/main.py", line 93, in
to_run = lambda: subcommand(args)
File "/home/bridgelan/miniconda3/envs/igdiscover/lib/python3.8/site-packages/igdiscover/cli/igblast.py", line 414, in main
logger.info('Processed {:10,d} sequences at {:.1f} ms/sequence'.format(n, elapsed / n * 1E3))
ZeroDivisionError: float division by zero
real 0m7,959s user 0m7,799s sys 0m0,575s [Sun Jul 4 21:17:56 2021] Error in rule igdiscoverigblast: jobid: 52 output: iteration-01/assigned.tab.gz, iteration-01/stats/assigned.json shell: time igdiscover igblast --sequence-type=Ig --rename 'testV3D0M' --threads=8 --stats=iteration-01/stats/assigned.json iteration-01/database reads/sequences.fasta.gz | pigz > iteration-01/assigned.tab.gz (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)
Removing output files of failed job igdiscover_igblast since they might be corrupted: iteration-01/assigned.tab.gz Shutting down, this might take some time. Exiting because a job execution failed. Look above for error message Complete log: /home/bridgelan/Documentos/testV3D0M/.snakemake/log/2021-07-04T211746.710417.snakemake.log Total CPU time: 0h 0.17m ERROR:
Before this error, I downloaded the most recent human database from IMGT, and I removed from the .fasta files all the alleles not present in the "human.ndm.imgt" file. I tried to run the test set with this database, and it worked.
The structure of my Ig sequences is: forward barcode - IGHV primer (beginning of V-REGION) - VDJC - IGHC primer (beginning of CH1 exon) - reverse barcode. I assembled the sequences and trimmed the barcodes and the primer regions before the IgDiscover run. As a precaution, I also inserted the primer sequences into the configuration file. Then, the message about trimming problems appeared, and I created the ./stats/trimming-successful file. Then, this error appeared.
I've analyzed this file with IgBlast before, and it seems non-corrupted: it detected 90.000 productive sequences in it. Is this too small for IgDiscover? And is this the error cause?
Thanks in advance.