search

jaraco / irc

Full-featured Python IRC library for Python.
MIT License
394 stars 86 forks source link

Library fuzzing #21

Closed jaraco closed 8 years ago

jaraco commented 8 years ago

ircfuzz is a nice little program written by Ilja van Sprundel, available here.

Basically, it throws (mostly) random data at a given client connection, and sees what breaks under load. This is useful if, for instance, somebody wants to shut your connection down and does this, crashing whatever process is running, or if a server legitimately (accidentally) sends you crap for whatever reason.

I'm wondering whether jaraco would be interested in me running through, fuzzing the hay out of this library, and tryna fix/point out whatever issues come up?

First off, here's one that crashed on my machine:

Raw message received by client from ircfuzz, split by irc's arguments:

    <type> <source> <target> <arguments>

    all_raw_messages danneh.riz None ['�J\x1a�:�q`Y�\x13�n�6�y�R9W�ةC��\x04��\x0e�!(i[\x1cڻu��L1R���*\x0b\x04���*!s�%\x1a�3��\\\x0fJx�\x06���9�&�����g\x7fӑ�F��`W�\x06FT\x15������k\U000c846f\x1e���y�>L�ޒ\x17��n����\x0fD�\x0e�v�FjA�\x19_Z�\x1eӫ\\\x1f9:�Pݥ�x�rh��%���t�\x05��\x1f\x14']

Traceback:

    File "/usr/lib/python3.3/site-packages/irc/client.py", line 264, in process_forever
        self.process_data(i)

        File "/usr/lib/python3.3/site-packages/irc/client.py", line 210, in process_data
            c.process_data()

            File "/usr/lib/python3.3/site-packages/irc/client.py", line 633, in process_data
                target = arguments[0]
jaraco commented 8 years ago

I can see addressing this issue in a couple of different ways:

One-Time Fuzzing

Run the fuzzer against the library, identify failure modes, file bugs, write tests, and fix the bugs.

Integrated Fuzzing Tests

We could integrate the fuzzing into the test suite such that tests are repeatedly run against random inputs (and could fail at some point in the future).

Other considerations

Some other things to consider:

  1. Do we test the client, the server, or both?
  2. Do we test the protocol, the command-line inputs, or both?
  3. Which configuration is used on the process (encoding/decoding config)? To some extent, the system is expected to fail on some inputs under certain configurations.

Overall, I feel like the value of this fuzzing is minimal, as it will under most circumstances only find bugs that will never be encountered. On the other hand, it might expose security vulnerabilities, which should be addressed. In the example above, I believe the behavior of the library is reasonable (raising a traceback when invalid arguments are passed on the command line) and not worth fixing.

Given these considerations, feel free to respond or put together a pull request if you believe the value justifies the effort.

Original comment by: Jason R. Coombs

jaraco commented 8 years ago

There's nothing to fix here, though I welcome specific reports or pull requests to improve the robustness of the library.

Original comment by: Jason R. Coombs