Threaded serve_forever

[afreeland]

Is there a way to run serve_forever asynchronously on its own thread. I am working on a Pi project that is utilizing websockets to easily stream sensor data to a web page. I am running into an issue with the serve_forever() function because it blocks everything else. Is there an easy way to thread it off to allow everything else to continue running?

Originally I had something along the lines of

def run():
    server.run_forever()
    # This code is never executed b/c server.run_forever
    while True:
        # Read photocell data
        light_level = readChannel(photocell)
        light_volts = convertVolts(light_level, 2)

I then tried something like this

def run():
    threading.Thread(target=server.run_forever()).start()
    # This still seems to block the thread...until  I hit 'Ctrl + C' which then appears
    # to allow code to continue executing and the socket server stays connected...
    while True:
        # Read photocell data
        light_level = readChannel(photocell)
        light_volts = convertVolts(light_level, 2)

Its weird that the threaded version allows me to hit Ctrl+C and the socket server appears to continue running and communicating with my webpage while letting my while statement run.

Is there a 'proper' way to thread this socket server that doesnt require me to hit Ctrl+C to allow code to continue?

[afreeland]

It appears that the Ctrl+C trick only works once the client already connects to the socket server

So this works

• start socket server
• connect browser client
• Ctrl+C python code
• execute remaining while loop

This does NOT work

• start socket server
• Ctrl + C
• connect browser client (fails)
• remaining loop runs

[afreeland]

Not quite sure what I did but will put it here if it helps anything...

Was looking at different way to do threading in python and ran across this threading link.

I went into the websocket_server.py file and modified

def run_forever(self):
        try:
            logger.info("Listening on port %d for clients.." % self.port)
            self.serve_forever()

to this

def run_forever(self):
        try:
            logger.info("Listening on port %d for clients.." % self.port)
            # self.serve_forever()
            _serve = self.serve_forever
            # Start a thread with the server -- that thread will then start one
            # more thread for each request
            server_thread = threading.Thread(target=_serve)
            # Exit the server thread when the main thread terminates
            server_thread.daemon = True
            server_thread.start()

Not sure why or how...but now my app appears to start the socket server...continue along peacefully and connect to my browser client. Best accidental discovery I have made today lol

Is this something that could be useful for others?

[murphy214]

@afreeland Dude you literally probably saved me hours with this thanks so much. Maybe you make pull request for it?

[afreeland]

@murphy214 Glad I could help =)

[Pithikos]

You don't have to do that. You should be able to run server_forever as a separate thread, similarly to how you did it.

I think the issue you first experienced was that you were passing threading.Thread(target=server.run_forever()) instead of (the correct) threading.Thread(target=server.run_forever). Notice that in the latter you are passing the function while in the first you are actually running the function.

[kirkatexo]

Original solution may have been overthinking it, but I had been trying something very similar with multiprocessing and running into pickle/cross-process issues with the server object. Didn't think to try threading instead until I saw this conversation. Kudos to @afreeland for the inadvertent solution.

And lest it be misinterpreted, many thanks to @Pithikos for the fantastic Pythonic WebSockets solution. Minimal dependencies, beautifully clean API, robust use-case support. Invaluable.