ashtree74 / MeshData

MeshData
1 stars 0 forks source link

Think about better file read and processing #3

Open kfigaj opened 6 years ago

kfigaj commented 6 years ago

https://github.com/ashtree74/MeshData/blob/73c7dbb765b86624fd49a95bdcbd0382d67effe1/main.py#L32

Think how this code would behave if your file would have millions of rows. One can do read and process more efficient - The simple solution here is to split data to chunks. Python support this concept with generators.

https://stackoverflow.com/questions/519633/lazy-method-for-reading-big-file-in-python https://campus.datacamp.com/courses/python-data-science-toolbox-part-2/bringing-it-all-together-3?ex=8#skiponboarding

ashtree74 commented 6 years ago

Looks like a great learning task. Thanks.