Too many open files: 'peercache.txt'

[xoriole]

Platform: Linux-4.10.0-42-generic-x86_64-with-Ubuntu-16.04-xenial Report: (#11881)

Traceback (most recent call last):
File "/usr/share/tribler/TriblerGUI/event_request_manager.py", line 99, in on_read_data
raise RuntimeError(json_dict["event"]["text"])
RuntimeError: [Failure instance: Traceback: <type 'exceptions.IOError'>: [Errno 24] Too many open files: u'/home/dsk/.Tribler/peercache.txt'
/usr/lib/python2.7/dist-packages/twisted/internet/base.py:1194:run
/usr/lib/python2.7/dist-packages/twisted/internet/base.py:1203:mainLoop
/usr/lib/python2.7/dist-packages/twisted/internet/base.py:825:runUntilCurrent
/usr/lib/python2.7/dist-packages/twisted/internet/task.py:239:__call__
--- <exception caught here> ---
/usr/lib/python2.7/dist-packages/twisted/internet/defer.py:150:maybeDeferred
/usr/share/tribler/Tribler/dispersy/discovery/community.py:801:clean_and_save
]

[devos50]

Tribler should handle open file descriptors in a smarter way. In this issue, it seems to be caused by a large number of open sockets due to HTTP requests to the torrent health checker.

[devos50]

Or we can simply have a queue of outstanding torrent health requests in the GUI.

[qstokkink]

We could make a request priority system for TriblerRequestManager.perform_request(), based on the number of unanswered requests (window).

Something like:

• CRITICAL: Send immediately
• HIGH: Send if current window size is < 80
• MEDIUM: Send if current window size is < 40, drop if window size > 200
• LOW: Send if current window size is < 20, drop if window size > 80

[qstokkink]

Actually, this would be a good opportunity to clean stuff like this:

https://github.com/Tribler/tribler/blob/16a1ea31a2a6e559ed425a87ab2280115bea56a9/TriblerGUI/tribler_request_manager.py#L45-L48

And refactor the queue logic into a single class.

[qstokkink]

If someone has the time, could you checkout https://github.com/qstokkink/tribler/tree/gui_request_queue and click all the things to see if something breaks (specifically by performing a lot of health requests)?

If you want to see the queue in action, change the following line: https://github.com/qstokkink/tribler/blob/ede9946ce8b3f4729a55781d12851f2bcb58a191/TriblerGUI/tribler_request_manager.py#L140 To: request_queue = RequestQueue(max_outstanding=5) And click a lot of torrents with unknown health.