Parse medaka model from raw Nanopore reads

[jmtsuji]

Newer Nanopore FastQ files (generated by Dorado?) seem to have the basecalling model information built into the FastQ header. We could try to parse this into the basecalling model format used by medaka so that the user doesn't need to input it manually. Different strategies are possible:

• parse this info while setting up the config, and add the parsed medaka model into the config file as a single variable
• determine this info on-the-fly while running rotary. This would allow you to analyze multiple samples having different medaka model info in the same rotary run, but the user would not have as much control over what is going on, and the pipeline would just error if one of the samples was missing the model info in the FastQ header. This option does not seem ideal.
• hybrid: while setting up the config, check all files for all samples, and if needed, specify the medaka model on a per-sample basis in the config file; otherwise set it globally in the config file

The last "hybrid" option is probably best -- what do you think @LeeBergstrand ?

[LeeBergstrand]

@jmtsuji Short term lets do it config wise. So if you're not providing a custom config it gets it from the first long fastq file. I'm not sure how likely it would be do a batch across flow cell or base caller versions. Long term lets do hybrid. One good option would be to add a new column to sample.tsv with this info but that might require edits to its parsing and generation code. I keep a note of that for snakemake utility library. You would have file path columns and metadata columns.

[jmtsuji]

@jmtsuji Short term lets do it config wise. So if you're not providing a custom config it gets it from the first long fastq file. I'm not sure how likely it would be do a batch across flow cell or base caller versions. Long term lets do hybrid. One good option would be to add a new column to sample.tsv with this info but that might require edits to its parsing and generation code. I keep a note of that for snakemake utility library. You would have file path columns and metadata columns.

OK, this sounds good as a roadmap. In the short-term, let's just parse the model from the first FastQ file. Long-term, that could potentially work well to add the medaka model to the samples.tsv file as metadata for each sample... I agree, it could be nice to keep this kind of future functionality in mind when you are writing the utility library.

[LeeBergstrand]

@jmtsuji https://github.com/nanoporetech/medaka#models

Medaka can now get the base caller model from data in the FASTQ file built-in. You still need to specify the model in the CLI for older files.

[jmtsuji]

This is excellent news! We might be able to use this in a few different ways:

• Easiest way: keep the medaka model setting in the config file as it is currently, but set the default to 'auto'. The medaka rule will just run without specifying a model name is 'auto' is set. Otherwise, the medaka rule will use the model provided by the user.
• More advanced way: try to auto-detect the medaka model name during init and add it to the samples.tsv file (or at least try to auto-detect if a modern Nanopore FastQ file format is used that includes the model name). Leave the model blank in the samples.tsv file if a model cannot be auto-detected

What are your thoughts about the best approach? The "easy way" could be implemented with minimal edits to the current code, even if we want to aim for the more advanced approach in future.

[LeeBergstrand]

Pursuing the easy way in https://github.com/rotary-genomics/rotary/pull/153

[LeeBergstrand]

Completed.