create cigar-based read filter

[akiezun]

feature request from @vdauwera (from https://github.com/broadinstitute/hellbender/issues/429) "Feature request: add ability to recognize a cigar pattern (to e.g. select reads with insertions> 10 bases, or reads with soft-clips, etc)."

@vdauwera please write an example commandline you'd like to be able to write (or a list of all patterns you want to be able to filter). Assign to me when done.

[vdauwera]

Ticket in gsa-unstable: https://github.com/broadinstitute/gsa-unstable/issues/832

If it gets implemented there we'll be sure to fwd-port to Hellbender as well.

[droazen]

Re-assigning to @jonn-smith, as this might be a fun one.

[jonn-smith]

@vdauwera can you provide some more examples of what kinds of cases you'd like to have handled?

[vdauwera]

There were some good basic examples in the original ticket:

• get all the contiguously aligned reads (e.g, xxM)
• get reads with soft clipping (e.g., xSxxM, could be reads with partial adapter sequence still left after trimming)
• get reads with insertions (e.g., xxMxxIxxM, could be spliced reads, e.g., reads spanning exon-exon, or intron-intron junction)
• get reads with deletions (g.g., xxMxxDxxM, could point at SV)

Those would be the basic must-haves.

Then the next step of nice-to-haves would be to be able to find specific patterns like "D followed by I" or specific numbers of operators like "exactly five D in a row" or "five D in total, not necessarily in consecutive order".

Do you need me to be more specific than that?

[jonn-smith]

@vdauwera - I think that makes sense. We've been brainstorming ideas for how a user would actually input the filter strings and there seem to be a few options.

• JEXL
  • it's already in use elsewhere and we can use JEXL functions at the command-line to specify "hasAtLeast(5,"D")" for simple filters, but it seems like it would get clunky with increasing filter complexity
• Regular Expressions
  • they're fairly universal, but it would be hard to match numerical values and can be confusing/exhausting to write correctly
• Modified regular expressions, for example:
  • ^D matches any number of deletions at the start
  • DMD matches any number of deletions followed by any number of (mis-)matches, followed by any number of deletions
  • ^<5SM>=4D$ matches less than 5 soft clipped bases at the start of the cigar, followed by any number of (mis-)matches, followed by at least 4 deletions at the end of the cigar
• Command-line options passed into the filter, for example:
  • --hasAtLeast 5 D --startsWith S --endsWith M

We can also implement some combination of these. What do you think?

[vdauwera]

I like the idea of the modified regexes, that seems like the best balance of usability and flexibility/power. I'd rather avoid having a slew of new special-cased arguments.