JEFworks-Lab / MERmaid

WebGL-based viewer for spatially-resolved transcriptomics data
http://jef.works/MERmaid/
26 stars 5 forks source link

use pako to read gzip compressed data #5

Closed slowkow closed 6 years ago

slowkow commented 6 years ago

The compressed data is about 3 times smaller in size:

61M data.csv 22M data.csv.gz

Steps:

  1. Use XMLHttpRequest() to get the data.csv.gz file.

  2. Use pako to decompress the data into a string.

  3. Use d3.csvParseRows() to read the string into an array.

I don't think you can show the progress with d3.csvParseRows().


To reduce the size of your git repo, you might consider following the steps in the link below to permanently delete the data.csv file. That will make it faster for others to git clone in the future, since they will avoid downloading the historical data.csv file.

https://help.github.com/articles/removing-sensitive-data-from-a-repository/