I have started using the library and specifically the gheto collect and consume methods the other day and i noticed that some tweets cannot get parsed, i applied the fix described under issue #53 but even with that some tweets are failing to get decoded into a valid json.
I debuged a bit the execution and i think the issue lies at the gheto consume and specifically at the area where the fgets is retrieving the json line. I believe that the line size of 8192 is not enough when fgets attempts to retrieve the json object and it gets partial json, this leads to incorrect json decode and tweets are getting lost.
let me know what you think, btw with a higher size i don't any loose tweets, now the right size is unknown to me.
Hi Guys,
I have started using the library and specifically the gheto collect and consume methods the other day and i noticed that some tweets cannot get parsed, i applied the fix described under issue #53 but even with that some tweets are failing to get decoded into a valid json.
I debuged a bit the execution and i think the issue lies at the gheto consume and specifically at the area where the fgets is retrieving the json line. I believe that the line size of 8192 is not enough when fgets attempts to retrieve the json object and it gets partial json, this leads to incorrect json decode and tweets are getting lost.
let me know what you think, btw with a higher size i don't any loose tweets, now the right size is unknown to me.