google / pasta

Library to refactor python code through AST manipulation.
Apache License 2.0
341 stars 40 forks source link

Support call-style exec in python 2.7.9+ #22

Closed soupytwist closed 6 years ago

soupytwist commented 6 years ago

Currently missing tests due to poor version separation. Need to test with the revision number as well, 2.7 is not specific enough. Will enable this separately.

soupytwist commented 6 years ago

@williamjamir, if you're able to, can you verify that this works for you on 2.7.9? I have some local configuring to do in order to get set up for many python versions. Currently I only test on 2.7.6 and 3.4.3.

williamjamir commented 6 years ago

Yep, with python 2.7.9 the parser identifies exec ('x = 10', {}) correctly

(google_pasta_py279) william@laptop:/opt/pasta$ python --version
Python 2.7.9 :: Continuum Analytics, Inc.
soupytwist commented 6 years ago

Thanks for the quick response!

soupytwist commented 6 years ago

Fixed #21.

williamjamir commented 6 years ago

@soupytwist just for a test I added the following test on the exec.in and it didn't pass.

exec(j,k)

___________________________________ SymmetricTest.test_symmetric_exec ___________________________________

self = <pasta.base.annotate_test.SymmetricTest testMethod=test_symmetric_exec>

    def test(self):
      with open(filepath, 'r') as handle:
        src = handle.read()
      t = ast_utils.parse(src)
      annotator = annotate.AstAnnotator(src)
      annotator.visit(t)
>     self.assertMultiLineEqual(codegen.to_str(t), src)
E     AssertionError: 'exec a\n\nexec b in c\n\nexec     d     in   e,f\n\nexec g in h   ,  i\n\nexecj [truncated]... != 'exec a\n\nexec b in c\n\nexec     d     in   e,f\n\nexec g in h   ,  i\n\nexec( [truncated]...
E       exec a
E       
E       exec b in c
E       
E       exec     d     in   e,f
E       
E       exec g in h   ,  i
E       
E     - execj,k+ exec(j,k)
E     ?     +   ++

pasta/base/annotate_test.py:129: AssertionError
=========================== 1 failed, 122 passed, 10 skipped in 0.50 seconds ============================
soupytwist commented 6 years ago

Thanks, reopened #21. I misunderstood your last message.

williamjamir commented 6 years ago

So, running the following script:

script.py:

import pasta
from pasta.augment import rename

path = r'test.py'

with open(path) as file:
        tree = pasta.parse(file.read())

where the content from test.py is the following: exec('x = 10, {})

Work just fine as you cann see above:

(google_pasta_py279) william@laptop:/opt/test_pasta$ python --version
Python 2.7.9 :: Continuum Analytics, Inc.
(google_pasta_py279) william@laptop:/opt/test_pasta$ python script.py 
(google_pasta_py279) william@laptop:/opt/test_pasta$ 

Whoever as a mentioned before, I thought that was necessary a test case on the exec.in file and added the following content locally:

exec(j,k)

And the test failed =/

(google_pasta_py279) william@laptop:/opt/pasta$ python --version
Python 2.7.9 :: Continuum Analytics, Inc.
(google_pasta_py279) william@laptop:/opt/pasta$ pytest 
======================================================= test session starts =======================================================
platform linux2 -- Python 2.7.9, pytest-3.3.2, py-1.5.2, pluggy-0.6.0
rootdir: /opt/pasta, inifile:
collected 133 items                                                                                                               

pasta/augment/import_utils_test.py .............                                                                            [  9%]
pasta/augment/rename_test.py ..........                                                                                     [ 17%]
pasta/base/annotate_test.py .................s.F..........s...s.........s................s............s...s.........s...    [ 86%]
pasta/base/ast_utils_test.py ....                                                                                           [ 89%]
pasta/base/scope_test.py ....s.s.......                                                                                     [100%]

============================================================ FAILURES =============================================================
________________________________________________ SymmetricTest.test_symmetric_exec ________________________________________________

self = <pasta.base.annotate_test.SymmetricTest testMethod=test_symmetric_exec>

    def test(self):
      with open(filepath, 'r') as handle:
        src = handle.read()
      t = ast_utils.parse(src)
      annotator = annotate.AstAnnotator(src)
      annotator.visit(t)
>     self.assertMultiLineEqual(codegen.to_str(t), src)
E     AssertionError: 'exec a\n\nexec b in c\n\nexec     d     in   e,f\n\nexec g in h   ,  i\n\nexecj [truncated]... != 'exec a\n\nexec b in c\n\nexec     d     in   e,f\n\nexec g in h   ,  i\n\nexec( [truncated]...
E       exec a
E       
E       exec b in c
E       
E       exec     d     in   e,f
E       
E       exec g in h   ,  i
E       
E     - execj,k+ exec(j,k)
E     ?     +   ++

pasta/base/annotate_test.py:129: AssertionError
======================================== 1 failed, 122 passed, 10 skipped in 0.45 seconds =========================================
(google_pasta_py279) william@laptop:/opt/pasta$ 

cc @nicoddemus