search

josiahseaman / FluentDNA

FluentDNA allows you to browse sequence data of any size using a zooming visualization similar to Google Maps. You can use FluentDNA as a standalone program or as a python module for your own bioinformatics projects.
65 stars 7 forks source link

Future: Just in time rendering #45

Open josiahseaman opened 6 years ago

josiahseaman commented 6 years ago

This feature is something I'll put off for now, but I know it's possible.

Given a fasta file, you could skip the entire image write and zoom stack steps which constitute about 80% of the total compute time. This would mean you could open a very large fasta file and immediately start browsing interactively.

HDF5 - random access file format - read in file, build index of where in memory to look, only render what's necessary. Could intercept OpenSeaDragon file requests and render those layers just in time. Would require sparse sampling of points (whole fasta in memory?)

I think realistically, this feature is lower priority than many types of annotation support.

hyanwong commented 6 years ago

For what it's worth, we are trying to move away from HDF5 and have started to use zarr (see e.g. http://alimanfoo.github.io/2016/04/14/to-hdf5-and-beyond.html). The author of zarr works just across the room from me, and is a very good programmer, so I trust this library.