Threading leak when refreshing active streams.

[fletom]

I noticed that after a certain amount of time running in auto-refresh mode, the program always errors out:

Traceback (most recent call last):
  File "/usr/local/bin/livestreamer-curses", line 9, in <module>
    load_entry_point('livestreamer-curses==1.4.0', 'console_scripts', 'livestreamer-curses')()
  File "/Library/Python/2.7/site-packages/livestreamer_curses/main.py", line 85, in main
    curses.wrapper(l)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/curses/wrapper.py", line 43, in wrapper
    return func(stdscr, *args, **kwds)
  File "/Library/Python/2.7/site-packages/livestreamer_curses/streamlist.py", line 217, in __call__
    self.run()
  File "/Library/Python/2.7/site-packages/livestreamer_curses/streamlist.py", line 301, in run
    self.check_online_streams()
  File "/Library/Python/2.7/site-packages/livestreamer_curses/streamlist.py", line 666, in check_online_streams
    pool = Pool(self.config.CHECK_ONLINE_THREADS)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/pool.py", line 714, in __init__
    Pool.__init__(self, processes, initializer, initargs)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/pool.py", line 159, in __init__
    self._repopulate_pool()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/pool.py", line 222, in _repopulate_pool
    w.start()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/dummy/__init__.py", line 75, in start
    threading.Thread.start(self)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 745, in start
    _start_new_thread(self.__bootstrap, ())
thread.error: can't start new thread

Sure enough, watching my system process monitor, the process' active thread count increases by the number of streams every time the refresh is run. It would seem that the calls to check online status are spawned as threads but not released afterwards. It eventually goes up to 1,000s of threads and then refuses to spawn any more.

[HarveyHunt]

I can confirm this too, using livestreamer-curses on Arch Linux

[gapato]

Hey, it should be fixed in 09f03d4a0d4944e0dce85aa95e68e7cf0e2b237f. Thanks for the helpful report, I actually had that issue too and didn't really know where to look. A quick check with

ps -L -o pid= -p <pid> | wc -l

seems to show a thread count of 1 when not refreshing, could you check on your side ?

[fletom]

That command always outputs 3 on my machine. The threading count I was observing came from Activity Monitor app on my Mac (the equivalent of a task manager) and I'm not sure what the equivalent is on the command line.

I installed @09f03d4 and I can confirm that the thread count now stays low no matter how much I refresh the streams. I doubt I'll see thread.error anymore but I'll let it run for longer and let you know if I do.

Thanks!

[gapato]

Ok! I'm preparing 1.5.0 and will release tomorrow if I don't hear from you by then.

[fletom]

It's been running all day without any threading errors. Definitely good for release. :)