search

epam / NGB

New Genome Browser (NGB) - a Web - based NGS data viewer with unique Structural Variations (SVs) visualization capabilities, high performance, scalability, and cloud data support
MIT License
161 stars 40 forks source link

Optimize S3 browsing performance #142

Open mike-miroliubov opened 6 years ago

mike-miroliubov commented 6 years ago

Currently, browsing files from S3 looks slow. We need to try using Amazon S3 SDK, it may be faster. To handle this we need to extend HTSJDK API and use S3 instead of plain HTTP, when using S3 files. We already have EnhancedUrlHelper.class, probably we need to additionaly extend it.

https://aws.amazon.com/ru/documentation/sdk-for-java/

brainstorm commented 5 years ago

Nowadays, wouldn't it make more sense to go for htsget instead?:

https://samtools.github.io/hts-specs/htsget.html

Namely:

https://github.com/EGA-archive/ega-htsget-client