Closed jangorecki closed 5 years ago
Did not helped for pandas.
Tried dtype
, engine="c"
, low_memory=True
. See https://github.com/pandas-dev/pandas/issues/22194 for more information
What might help in future is https://github.com/h2oai/datatable/issues/1691
as documented in https://github.com/h2oai/db-benchmark/issues/111 currently pandas and dask fails on 1e9 data when attempting to read csv. try those, maybe will help https://stackoverflow.com/a/27232309/2490497 https://www.dataquest.io/blog/pandas-big-data/