Loading large csv data - Githubissues

fslaborg / Deedle

Easy to use .NET library for data and time series manipulation and for scientific programming

http://fslab.org/Deedle/

BSD 2-Clause "Simplified" License

935 stars 195 forks source link

Closed Radibor closed 5 years ago

Radibor commented 10 years ago

Loading large csv data seems into a Deedle frame seems to cause memory problems.

On my laptop, I was not able to load more than 200 MB.

train_set.csv from

is 2.5 GB and can be used as a test case.

zyzhu commented 5 years ago

Tested on the 2.6GB train_set.csv. It took a while to finish on FSI and FSI took 13.5GB memory.