MichaelCurrin / python-twitter-guide

Code snippets and links to docs around using the Twitter API and Tweepy 🐍 🐦
https://michaelcurrin.github.io/python-twitter-guide/
MIT License
4 stars 8 forks source link

Add streaming fix #41

Open MichaelCurrin opened 4 years ago

MichaelCurrin commented 4 years ago

From https://stackoverflow.com/questions/26638329/incompleteread-error-when-retrieving-twitter-data-using-python

Found through https://github.com/Kydlaw/pumpy/issues/1#issuecomment-610553673

When you get a disconnect from urllib ProtocolError, a solution is to add threading so

from queue import Queue
from threading import Thread

class Listener(tweepy.StreamListener):

    def __init__(self, q = Queue()):
        super().__init__()
        self.q = q

        for i in range(4):
            t = Thread(target=self.do_stuff)
            t.daemon = True
            t.start()

    def on_status(self, status):
        print(status)

    def do_stuff(self):
        while True:
            self.q.get()
            self.q.task_done()