Crash in case of huge pile

medialab / gazouilloire

Twitter stream + search API grabber

GNU General Public License v3.0

104 stars 17 forks source link

Crash in case of huge pile #146

Open bmaz opened 1 year ago

bmaz commented 1 year ago

Should we avoid loading the pile if the file is too big?

bmaz commented 1 year ago

Too big stack files could be solved by saving them by packets of say 50,000 tweets and unpacking them at reload one by one before launching the collections