jazzband / tablib

Python Module for Tabular Datasets in XLS, CSV, JSON, YAML, &c.
https://tablib.readthedocs.io/
MIT License
4.61k stars 593 forks source link

Dataset can read/load only first n rows of csv file. #467

Closed sandeepsajan0 closed 2 years ago

sandeepsajan0 commented 4 years ago

If we have a huge csv file and we can test our code only by first n rows then we shouldn't need to load the whole file. It'll be a great feature. I tried to find it but couldn't if it already exists there, can anyone of you guys tell me how can I use that feature? Or we should implement that feature?

**Edited: For now, I am just using the list comprehension with del method on loaded data to pass. But it'll be better if I can load only desired data. What are your ideas?

mynameisvinn commented 4 years ago

@sandeepsajan0 the common solution is to read/parse csv file with a generator, so you can iterate over your csv lazily without loading the entire file into memory (and allowing you to peek the first n files).

hugovk commented 2 years ago

Closing, as the solution is to iterate lazily using a generator.