search

nalmadi / EMIP-Toolkit

EMIP Toolkit (EMTK): A Python Library for Processing Eye Movement in Programming Data
16 stars 22 forks source link

adding download & unzipping progress bar & download as stream #49

Closed luhangsnn closed 2 years ago

luhangsnn commented 2 years ago

(https://github.com/nalmadi/EMIP-Toolkit/issues/34) added downloading and unzipping progress bar to the download method that displays the file size, chunk size, and estimated finishing time.

image minor issue for unzipping: when running on jupyter notebook, the output will first print a separate progress bar that will end at ~1% and then the proper progress bar for unzipping. This issue does not happen when testing in powershell terminal - have not found a solution to this jupyter-specific issue yet

I believe it also splits the download into chunks - 1024 bytes at a time (download as stream?) so that we don't need to hold the whole dataset in memory when downloading. with requests.get(url, stream=True) as r then for chunk in r.iter_content(chunk_size=1024):

nalmadi commented 2 years ago

Thank you! You made a 2nd pull-request so I am closing this one.