On recent Linux versions, commands.process always crashes if a heap size lower than ~1GB is given. - Githubissues

progval / Limnoria

A robust, full-featured, and user/programmer-friendly Python IRC bot, with many existing plugins.

https://docs.limnoria.net/

Other

623 stars 173 forks source link

On recent Linux versions, commands.process always crashes if a heap size lower than ~1GB is given. #1271

Open weedy opened 7 years ago

weedy commented 7 years ago

Exception in thread Thread #2 (for snarfing http://imgur.com/gallery/DQlH8):
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
    self.run()
  File "/home/weedy/supybot/lib/python2.7/site-packages/supybot/commands.py", line 164, in run
    super(UrlSnarfThread, self).run()
  File "/usr/lib/python2.7/threading.py", line 754, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/home/weedy/supybot/lib/python2.7/site-packages/supybot/commands.py", line 215, in doSnarf
    f(self, irc, msg, match, *L, **kwargs)
  File "/home/weedy/supybot/lib/python2.7/site-packages/supybot/plugins/Web/plugin.py", line 109, in newf
    for (method, args, kwargs) in replies:
TypeError: 'NoneType' object is not iterable

progval commented 7 years ago

What is your Linux version (the kernel, not the distribution)?

weedy commented 7 years ago

4.8.6

progval commented 7 years ago

This is going to sound funny, but I believe this problem happens on recent versions of Linux.

So far, I have been able to determine that 3.16 is fine, but 4.7 is not.

progval commented 7 years ago

This does not seem to be a Limnoria-specific issue, so I forwarded it to CPython's bugtracker: https://bugs.python.org/issue28663

However, this bug does affect Limnoria, so I'm keeping this issue open, and will try to find a solution.

weedy commented 7 years ago

You made the bug for 3.x versions. I'm running 2.7.

progval commented 7 years ago

That's because Python 2.7 no longer receives updates.

dequis commented 7 years ago

Why file a bug against python if it's clearly a kernel regression?

progval commented 7 years ago

It may be a bad behavior of CPython. If not, they can forward the bug to Linux.

progval commented 7 years ago

So. This bug is caused by the “sandbox” feature of Limnoria, which prevents process from eating all of the memory. The Web plugin uses this sandbox in case a web server to which the bot connects from flooding the bot with HTTP headers.

However, I just looked at Python's HTTP library, and it fixed this vulnerability in 2014. I just pushed a commit (9c57199838ed167ff4d810a91f489eb32bd717f0) to testing, so that if your Python version is not vulnerable (see the list of vulnerable versions), then the sandbox will not be used, and you will not have this bug.

However, this does not completely fixes the bug, because there may be other use cases of this sandbox. So I would like to keep this issue open.