statgen / pheweb

A tool to build a website to browse hundreds or thousands of GWAS.
MIT License
158 stars 65 forks source link

Gzip outgoing data #11

Closed pjvandehaar closed 8 years ago

pjvandehaar commented 8 years ago

With the pval_threshold 1e-4 and bin sizes 0.05 and 3e6, the Manhattan Plot data is ~1MB. (2/3 unbinned_variants and 1/3 variant_bins.) Gzipped, it's <100KB. (4/5 unbinned_variants and 1/5 variant_bins.)

Option 1: Just use Flask-Compress.

Option 2: For maximum speed, we'll pre-gzip (-9!) our static files and directly serve them. The headers for gzipped data can be found at http://flask.pocoo.org/snippets/122/. https://spoqa.github.io/flask-s3/_modules/flask_s3.html might also be interesting. But how do I attach headers with send_from_directory or send_file (which it uses)? They return responses, right? Be sure to pass a long cache_timeout and to handle the mimetype guessing.

pjvandehaar commented 8 years ago

I'm using Flask-Compress. Static compression would be more interesting, but this is good enough for now.