search

matthiasn / BirdWatch

Tweet stream analysis and visualization with real-time updates.
http://matthiasnehlsen.com/
GNU Affero General Public License v3.0
856 stars 153 forks source link

Process chunk from buffer in final step when assembling tweet chunks #29

Open sandhu opened 9 years ago

sandhu commented 9 years ago

In the final step the buff will contain the last chunk. The final step now ensures that the left over chunk is processed.

matthiasn commented 9 years ago

Hey Sandhu, thank you for the pull request and sorry for not getting around to it sooner. I will check it later this week and see if I can merge it. However, this condition doesn't really appear where the left over chunk needs to be processed does not really occur because there's an ongoing stream. When the application is closed, it hardly matters if there's probably another full item in there. Even if, when I quit I'm going to miss data from some point on. The application is really meant to be kept running. Cheers, Matthias

sandhu commented 9 years ago

No worries on the delay Matthias. I definitely agree with your comment that the issue will not be observed in the scenario the transducer is being used in this particular application, however if you wanted to use this transducer with a vector you'd see the issue.

Since the transducer represents an abstracted computation outside the specific scenario of a particular application to data, I created the PR.

Cheers, Achint