SyntaxError in yapflib.verifier

[DRMacIver]

The following code:

class YapfTest(unittest.TestCase):
  def testSimple(self):
    unformatted_code = textwrap.dedent(u"""\
        print('foo')
        """)

Gives the following error:

INTERNAL ERROR: 
        unformatted_code = textwrap.dedent(u"""\
        print('foo')
        """)
Traceback (most recent call last):
  File "/home/david/yapf/yapf/yapflib/verifier.py", line 38, in VerifyCode
    compile(textwrap.dedent(code).encode('UTF-8'), '<string>', 'exec')
  File "<string>", line 4
    """)
      ^
SyntaxError: invalid syntax

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/david/yapf/yapf/yapflib/verifier.py", line 41, in VerifyCode
    ast.parse(textwrap.dedent(code.lstrip('\n')).lstrip(), '<string>', 'exec')
  File "/usr/lib/python3.2/ast.py", line 36, in parse
    return compile(source, filename, mode, PyCF_ONLY_AST)
  File "<string>", line 3
    """)
      ^
SyntaxError: invalid syntax

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.2/runpy.py", line 160, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python3.2/runpy.py", line 73, in _run_code
    exec(code, run_globals)
  File "/home/david/yapf/yapf/__main__.py", line 18, in <module>
    sys.exit(yapf.main(sys.argv))
  File "/home/david/yapf/yapf/__init__.py", line 102, in main
    print_diff=args.diff)
  File "/home/david/yapf/yapf/__init__.py", line 124, in FormatFiles
    filename, style_config=style_config, lines=lines, print_diff=print_diff)
  File "/home/david/yapf/yapf/yapflib/yapf_api.py", line 67, in FormatFile
    print_diff=print_diff)
  File "/home/david/yapf/yapf/yapflib/yapf_api.py", line 110, in FormatCode
    reformatted_source = reformatter.Reformat(uwlines)
  File "/home/david/yapf/yapf/yapflib/reformatter.py", line 73, in Reformat
    verifier.VerifyCode(formatted_code[-1])
  File "/home/david/yapf/yapf/yapflib/verifier.py", line 45, in VerifyCode
    compile(normalized_code.encode('UTF-8'), '<string>', 'exec')
  File "<string>", line 3
    """)
      ^
SyntaxError: invalid syntax

...as you can probably tell, by this point I have a fuzzer doing most of the work for me (about the first half + a few of the subsequent ones have been hand produced) and this is from turning it on the yapf codebase itself. If you want it I'd be happy to give you a copy of the fuzzer.

[bwendling]

I'm not able to replicate this bug. I appreciate the reports. I'd be interested in looking at the fuzzer. :-)

[morbo:yapf] python ./build/lib/yapf --diff /tmp/bad.py
--- /tmp/bad.py (original)
+++ /tmp/bad.py (reformatted)
@@ -1,5 +1,5 @@
 class YapfTest(unittest.TestCase):
-  def testSimple(self):
-    unformatted_code = textwrap.dedent(u"""\
+    def testSimple(self):
+        unformatted_code = textwrap.dedent(u"""\
         print('foo')
         """)

Is there some command line option that I'm missing?

[DRMacIver]

Ah! Apologies. This is only reproducible on python 3.2 it turns out (possibly others, but it works correctly on python 2.7 and python 3.4, I don't have any others readily to hand)

[DRMacIver]

Here's the current copy of the fuzzer: https://gist.github.com/DRMacIver/a10e2624e5529b70d881

Note embedded disclaimer about how it's terrible. You probably don't want to actually use this code :-)

[DRMacIver]

Given that you don't support 3.2, I'll close this as invalid. I've patched the fuzzer to let me set the python version properly. :-)