Escaping docstrings

[rocky]

Here is a bug in how docstrings are handled...

r'''func placeholder - with ("""\nstring\n""")'''
def foo():
    r'''func placeholder - ' and with ("""\nstring\n""")'''

def bar():
    r"""func placeholder - ' and with ('''\nstring\n''') and \"\"\"\nstring\n\"\"\" """

def baz():
    """
        ...     '''>>> assert 1 == 1
        ...     '''
        ... \"""
        >>> exec test_data in m1.__dict__
        >>> exec test_data in m2.__dict__
        >>> m1.__dict__.update({"f2": m2._f, "g2": m2.g, "h2": m2.H})

        Tests that objects outside m1 are excluded:
        \"""
        >>> t.rundict(m1.__dict__, 'rundict_test_pvt')  # None are skipped.
        TestResults(failed=0, attempted=8)
    """

Compiling/decompiling gives:

Traceback (most recent call last):
  File "unpyc37/unpyc3.py", line 2901, in <module>
    print(decompile(sys.argv[1]))
  File "unpyc37/unpyc3.py", line 1507, in __str__
    self.display(istr)
  File "/unpyc3.py", line 1513, in display
    stmt.display(indent)
  File "/unpyc3.py", line 1337, in display
    self.display_undecorated(indent)
  File "/unpyc3.py", line 1362, in display_undecorated
    DocString(docstring).display(indent + 1)
  File "unpyc3.py", line 1153, in display
    raise NotImplemented
TypeError: exceptions must derive from BaseException

[andrews4s]

Hey rocky, the exception should be fixed now in 7ed40f1dd7c1f3e4ffc68ebcef1c5cfc28411d56. I added your test case without the raw string literal syntax though. If that can affect the byte code and should be handled, let me know.

[rocky]

~I believe it should be, for a very simple reason: you can take that program and run it using a Python 3.7 interpreter and it runs. So everything is valid.~

Edit I was wrong above, what you have is correct.

Here is a better test program which shows what you have is correct:

r'''func placeholder - with ("""\nstring\n""")'''
def foo():
    r'''func placeholder - ' and with ("""\nstring\n""")'''

def bar():
    r"""func placeholder - ' and with ('''\nstring\n''') and \"\"\"\nstring\n\"\"\" """

def baz():
    """
        ...     '''>>> assert 1 == 1
        ...     '''
        ... \"""
        >>> exec test_data in m1.__dict__
        >>> exec test_data in m2.__dict__
        >>> m1.__dict__.update({"f2": m2._f, "g2": m2.g, "h2": m2.H})

        Tests that objects outside m1 are excluded:
        \"""
        >>> t.rundict(m1.__dict__, 'rundict_test_pvt')  # None are skipped.
        TestResults(failed=0, attempted=8)
    """
    assert __doc__ == r'''func placeholder - with ("""\nstring\n""")'''
    assert foo.__doc__ == r'''func placeholder - ' and with ("""\nstring\n""")'''
    assert bar.__doc__ == r"""func placeholder - ' and with ('''\nstring\n''') and \"\"\"\nstring\n\"\"\" """
    assert baz.__doc__ == \
    """
        ...     '''>>> assert 1 == 1
        ...     '''
        ... \"""
        >>> exec test_data in m1.__dict__
        >>> exec test_data in m2.__dict__
        >>> m1.__dict__.update({"f2": m2._f, "g2": m2.g, "h2": m2.H})

        Tests that objects outside m1 are excluded:
        \"""
        >>> t.rundict(m1.__dict__, 'rundict_test_pvt')  # None are skipped.
        TestResults(failed=0, attempted=8)
    """

baz()

When you byte-compile this, then run the decompiled results from unpyc37, this works, so your fix is truly a fix. I note that using the output from uncompyle6 though, that resulting program gives an assertion error, so that code has a bug which needs to be fixed.

I'll open the next issue going over test https://github.com/rocky/python-uncompyle6/tree/master/test/bytecode_3.7 soon.

One last thing... that complicated doc string in fact comes from one of the programs in one of the Python standard libraries. Decompiling everything in a standard library is a great way to find bugs.