Implement same user detection and threading in a better way

Currently a dictionary is used to store users currently being processed. Also, threads are created within threads. Instead of creating a Thread(), use concurrent.futures.ThreadPoolExectuor to run the response function, adding the futures object to a dictionary. Then, instead of sending messages from within the function, return the message. Then we can loop through concurrent.futures.as_completed(DICT{FUTUREOBJECT: SENDER, ....}) with a timeout of something like .05s or something similarly low. Then the object returned through the loop .result() will contain the message, and we can send that. This will also be done in the main thread, so sleeps will actually work to throttle down sending.

Or we can use add_done_callback(fn), where fn is a method that takes in a future object. However, this would require throttling a different way. A simple way would be remove the sender from senders dict inside the fn method (which would pretty much just be a method calling future.result() then calling send)

import concurrent.futures
import urllib.request

URLS = ['http://www.foxnews.com/',
        'http://www.cnn.com/',
        'http://europe.wsj.com/',
        'http://www.bbc.co.uk/',
        'http://some-made-up-domain.com/']

def load_url(url, timeout):
    return urllib.request.urlopen(url, timeout=timeout).read()

with concurrent.futures.ThreadPoolExecutor(max_workers=5) as executor:
    future_to_url = dict((executor.submit(load_url, url, 60), url)
                         for url in URLS)
    print(future_to_url)
    for future in concurrent.futures.as_completed(future_to_url, timeout=0.05):
        url = future_to_url[future]
        if future.exception() is not None:
            print('%r generated an exception: %s' % (url,
                                                     future.exception()))
        else:
            print('%r page is %d bytes' % (url, len(future.result())))
        del future_to_url[future]
    print(future_to_url)

ApolloFortyNine / SongSense

Implement same user detection and threading in a better way #75