bcbio priorities

[naumenko-sa]

Long TODO lists don't get done. We are trying to keep an issue open if a real effort is happening to resolve it. The below priority list gives a sense of where bcbio is heading in the next two releases.

Issues from the lower circles are re-opened and pushed to circle one, when there is bandwidth to work on them.

Contributions are welcome to rescue an issue from any circle!

Closed issues and implemented features are described in the release notes.

Priority circle 1 ~ ~ release 1.3.0 ~ Q2 2022

• [ ] update transcriptome gtf from ensembl 94 to ensembl 104 - TSL1-3

Priority circle 2

• [ ] WGBS: + qualimap bamqc
• [ ] somalier for detecting mislabelled samples
• [ ] more tests of bcbio-se.R
• [ ] output a JSON file describing output files in the final for downstream uploader systems (solvebio)
• [ ] test and implement output of BigWig files in RNA-seq and ATAC-seq project (motivation: not everyone can store all bam files)
• [ ] add low level (barcodes, %rRNA) QC to single cell pipeline
• [ ] explore implementation of ichorCNA
• [ ] make consistent use of conda/mamba for bcbio_nextgen_install.py and bcbio_nextgen.py upgrade
• [ ] make an option "'--very-sensitive" instead of "--very-sensitive-local' in chip-seq. see https://github.com/bcbio/bcbio-nextgen/pull/3476
• [ ] --funsegmentation GATK4 in purecn.R for exomes
• [ ] CWL #2904 #2715 #2872 #3471
• [ ] better gatkcnv support - cnvcalling with PON, gene panels seq and WES #3227
• [ ] Variant calling in RNA-seq - validation #3161
• [ ] Multiqc documentation improvement ; related: #3075
• [ ] Latest STAR release 2.7.0d requires genome index rebuild #2693 (discuss)
• [ ] Dfkz filter #3202
• [ ] Mosdepth index files missing #3124
• [ ] Validation of ALT contig calls #3069
• [ ] multiqc for T/N
• [ ] cfDNA calling: test and integrate ichor #2254
• [ ] SV validation
• [ ] SV annotation #963
• [ ] Integrate Gnomad SV data and methods
• [ ] DNA/RNA variant calling
• [ ] T/N RNA-seq #2521 #3023
• [ ] update germline & somatic validation #3038 #2921 use data from https://www.biorxiv.org/content/10.1101/2021.02.27.433136v1.full.pdf
• [ ] PON support for mutect2 #3254

Priority circle 3

new functionality

• [ ] Add passing in known fusions #3030
• [ ] GATK4 GermlineCNVCaller #2245
• [ ] AWS batch #3207 #3022
• [ ] make deep variant work in bcbio #3187 #3012
• [ ] Duplicates % from bamsormadup in QC
• [ ] Unaligned bam/cram (GI)
• [ ] Support for projects where samples are added to cohort #3036 #2336
• [ ] Small RNA-seq up and running
• [ ] Switch to Salmon selective alignment method #3166
• [ ] Mosiac chromosomal alterations #3113
• [ ] miRNA pipeline #3103
• [ ] validate and improve use of Octopus #1591
• [ ] document and improve use of GRIDSS for genomic rearrangements #1829
• [ ] Integrate vcf2maf #2293
• [ ] call repeat expansions #2797
• [ ] interchromosomal translocations with delly #2793
• [ ] RFE: svprioritize for multiple lists #2325
• [ ] Including piRNA quantification in smallRNAseq pipeline #2439
• [ ] vcfanno and gemini for RNA-seq variants #2511
• [ ] Single-cell pipeline drops FASTA spike-ins if not defined in GTF file #2387
• [ ] Support for Smoove/Lumpy population SV calling #2652
• [ ] Option to add -I -I flag in Arriba #3018
• [ ] Are there any Cut & Run analysis specific parameters implemented in bcbio? #3037
• [ ] mapDamage on bcbio #2867
• [ ] alignment-only RNA-seq pipeline #3239 (discuss)
• [ ] log running time of individual tools #3253
• [ ] support rsem to re-run older projects #3276
• [ ] consider strelka --rna for RNA-seq variant calling #3319
• [ ] productionalize purity/ploidy analysis #2532
• [ ] LoFreq #3322

Priority circle 4

Technical debt

The issues below don't break production, however, it is good to have them solved at some point. Contributions welcome - pick an issue, ask to re-open the issue if you are working on it. Some of them are good for the first contribution to bcbio.

• [ ] refactoring of picard usage https://github.com/bcbio/bcbio-nextgen/issues/3553
• [ ] VEP annotations can have a SOMATIC field and this breaks vcf2db parsing #2966
• [ ] conflicting annotations from VEP and gnomad #2945
• [ ] Using DP to extract somatic AF in strelka2.py might be wrong #2836
• [ ] Identical miRNA read counts versus multimapping #2893
• [ ] Configuration check should not allow for duplicated "description" items #2738
• [ ] update citations #2125
• [ ] test tools against python3.7 and switch bcbio to python3.7 with dataclasses
• [x] clinvar UnicodeDecodeError: pybedtools issue #3078, waiting for pybedtools 0.8.2
• [ ] CnvKit - 0.9.7b issues #3201 - for new cnvkit 0.9.7 release
• [ ] Error in "Getting Started" instructions #3162
• [ ] Remove Sailfish support #3157
• [ ] Speed up installation of third party tools #3238 related to
• [ ] Make installation and upgrading more robust #3191
• [ ] TypeError in test_atacseq #3225
• [ ] FileNotFoundError in test_chipseq #3224
• [ ] Allow custom genome installation with --no-data #3226
• [ ] ValueError in test_variantcall_2 #3218
• [ ] Value error in test_detect_fusions_with_ericscipt_with_disambiguate #3220
• [ ] ValueError in test_detect_fusions_with_ericscipt_without_disambiguate #3219
• [ ] ValueError in test_variantcall_1 #3217
• [ ] ValueError in test_fusion #3216
• [ ] Error running bcbio_nextgen.py version sub-command #3209
• [ ] Upgrade to Ubuntu 20.04 LTS #3192
• [ ] Move Docker tests out of the bcbio-nextgen repo #3176
• [ ] Lock conda environments #3174
• [ ] Avoid editing the source code to keep the work directory between test runs #3155
• [ ] Remove legacy AWS setup documentation and code #3134
• [ ] Remove legacy python2 code (see https://github.com/bcbio/bcbio-nextgen/issues/3247#issue-626855618)
• [ ] Drop the incorrect genoype attributes for decomposes SNP's. #1334
• [ ] test cancermulti fails because strelka wants /usr/bin/env python2 #3019
• [ ] Don't require a dummy genome when transcriptome_fasta and transcriptome_gtf are specified #3264

Reference issues

Closed issues where no work is needed but which contain some valuable information (validations, great expertise and discussion). for better visibility.

• Validations in GRCh37 w and without decoy + RR #2489
• Pipeline standardization #2260
• Large DEL validations with HuRef benchmark #2313
• variant annotation for WGS #2587
• RFC: Tumor purity/ploidy estimation methods with targeted panels #2639
• 'add_log2_depth' issue during CNV analysis (CNVkit) #3004
• Which variant callers are compatible to TitanCNA? #2975
• Support for lossy CRAM in bcbio WGrS variant calling pipeline #2531
• RFE: Use germline resources in MuTect2 to reduce artifacts in tumor-only mode #2873
• Encoding errors with CIVIC svprioritize and IPython distributed tasks #2890
• a hint on somatic variants prioritization #3418

[roryk]

If anyone wants to tag in and work on any of this stuff, we are happy to help orient.

[lbeltrame]

cfDNA calling: test and integrate ichor

I can help with this, as I've used the package extensively (200+ samples so far). It would probably need some tweaking for low fraction samples.

On cfDNA front it might be worth investigating in silico size selection, which in my tests improves sensitivity a lot. The rationale is explained in https://pubmed.ncbi.nlm.nih.gov/30404863/, which describes a dual approach (both during library preparation and during analysis), but in my experience just filtering BAMs by query size length helps. I have proof-of concept code that does this step, which we use internally.