Failure when decompiling a conditional generator, python 2.5

[codepainters]

Current version of uncompyle2 fails to handle the following piece of code compiled with Python 2.5:

a=[1,2,3]
b=(1 for i in a if i)

The corresponding bytecode:

  3           0 SETUP_LOOP              29 (to 32)
              3 LOAD_FAST                0 (.0)
        >>    6 FOR_ITER                22 (to 31)
              9 STORE_FAST               1 (i)
             12 LOAD_FAST                1 (i)
             15 JUMP_IF_FALSE            9 (to 27)
             18 POP_TOP             
             19 LOAD_CONST               0 (1)
             22 YIELD_VALUE         
             23 POP_TOP             
             24 JUMP_ABSOLUTE            6
        >>   27 POP_TOP             
             28 JUMP_ABSOLUTE            6
        >>   31 POP_BLOCK           
        >>   32 LOAD_CONST               1 (None)
             35 RETURN_VALUE   

Assembler from uncompyle -a:

0   SETUP_LOOP        '30'
3   LOAD_FAST         '.0'
6   FOR_ITER          '29'
9   STORE_FAST        'i'
12  LOAD_FAST         'i'
15  JUMP_IF_FALSE     '26'
18  LOAD_CONST        1
21  YIELD_VALUE       None
22  POP_TOP           None
23  JUMP_BACK         '6'
26  JUMP_BACK         '6'
29  POP_BLOCK         None
30_0    COME_FROM         '0'
30  LOAD_CONST        None
33  RETURN_VALUE      None

There are 2 issues here:

• scanner for Python 2.5 is supposed to strip away SETUP_LOOP and POP_BLOCK, but it fails to do it
• there's no grammar production with double JUMP_BACK

[codepainters]

Note: the following hack made it work for me (yet the code is fairly complex, I'm not sure if it doesn't break anything):

diff --git a/uncompyle2/Parser.py b/uncompyle2/Parser.py
index 89fc675..3aced10 100755
--- a/uncompyle2/Parser.py
+++ b/uncompyle2/Parser.py
@@ -136,6 +136,7 @@ class Parser(GenericASTBuilder):
         stmt ::= genexpr_func

         genexpr_func ::= LOAD_FAST FOR_ITER designator comp_iter JUMP_BACK
+        genexpr_func ::= LOAD_FAST FOR_ITER designator comp_iter JUMP_BACK JUMP_BACK
         '''

diff --git a/uncompyle2/Scanner25.py b/uncompyle2/Scanner25.py
index 8e74051..9fc3f4d 100755
--- a/uncompyle2/Scanner25.py
+++ b/uncompyle2/Scanner25.py
@@ -321,8 +321,12 @@ class Scanner:
             if self.code[i+3] == LOAD_FAST and self.code[i+6] == FOR_ITER: 
                 end = self.first_instr(i, len(self.code), RETURN_VALUE)
                 end = self.first_instr(i, end, YIELD_VALUE)
-                if end and self.code[end+1] == POP_TOP and self.code[end+2] == JA and self.code[end+5] == POP_BLOCK:
-                    return [i,end+5]
+               # At this point 'end' points to YIELD_VALUE
+                if end and self.code[end+1] == POP_TOP and self.code[end+2] == JA:
+                    if self.code[end+5] == POP_BLOCK:
+                        return [i,end+5]
+                    elif self.code[end+5] == POP_TOP and self.code[end+6] == JA and self.code[end+9] == POP_BLOCK:
+                        return [i,end+9]
         # with stmt
         if opcode == WITH_CLEANUP:
             chckDel = i-self.op_size(DELETE_NAME)

[Mysterie]

This is patched. The grammar handle bytecode version 2.7 so I won't change it. But I've change bytecode 2.5 & 2.6 disassembly and translation.

[codepainters]

Yeah, I understand that - yet for me the quickest solution was to tweak the grammar :)

Is your solution to delete the second POP_TOP/JA pair?

[Mysterie]

In your case I delete :

0   SETUP_LOOP        '30'
26  JUMP_BACK         '6'
29  POP_BLOCK         None

And the FOR_ITER has to be modificated. But I don't do this in all case. If I do, some FOR loop won't work anymore. I apply the patch only when there is no GET_ITER between SETUP_LOOP and FOR_ITER instruction.

            # conditional tuple
            if self.code[i] == JA and self.code[i+opsize] == POP_TOP \
                and self.code[i+opsize+1] == JA and self.code[i+opsize+4] == POP_BLOCK:
                jmpabs1target = self.get_target(i)
                jmpabs2target = self.get_target(i+opsize+1)
                if jmpabs1target == jmpabs2target and self.code[jmpabs1target] == FOR_ITER:
                    destFor = self.get_target(jmpabs1target)
                    if destFor == i+opsize+4:
                        setupLoop = self.last_instr(0, jmpabs1target, SETUP_LOOP)
                        standarFor =  self.last_instr(setupLoop, jmpabs1target, GET_ITER)
                        if standarFor == None:
                            self.restructJump(jmpabs1target, destFor+self.op_size(POP_BLOCK))
                            toDel += [setupLoop, i+opsize+1, i+opsize+4]

I've check the whole python standard library and this solution work well. But I think tuple loop detection can be more efficient.

[NadhiyaAnbu]

how to convert pyo and pyc files to py file using uncompyle2 in python2.7 for windows xp

[Ophidiophobia]

read https://github.com/Mysterie/uncompyle2#usage

[NadhiyaAnbu]

@Ophidiophobia thank you. I installed unompyle2-master according to the steps given in https://github.com/Mysterie/uncompyle2#usage . In that i dont know where to enter the usage comment (uncompyle2 --help)

Then i used uncompyle2 to convert pyo file to py file in following steps:

usr/bin/folder/xyz.pyc > xyz.py (this comment is producing a empty py file and i am getting a runtime error:bad magic number in .pyc)

Please suggest solution for this :)

[Mysterie]

I don't understand what you're trying to do. But for using uncompyle2 you just have to do ./uncompyle2.py /path/xyz.pyc. The windows version of the projet isn't perfect for the moment.

The bad magic number suggest that it's not a valid pyc file.

PS: can you open a new issue, or your bug is related to the conditional tuple patch ?

[Ophidiophobia]

1. you are already hijacking the comments of one issue for your own problem. Please open a new issue if you have a new problem.
2. I do not own a crystal ball, I do not know what you see or what you have done. What I know so far: You do no research (else you would have found the main page or read the file called README using a text editor of your choice) otherwise you wouldn't have asked your first question. At least not the way how you asked it. You do not know how to use python. Al is done on the console. To change a python script you can use a text editor or a python IDE (development tool of your choice). Advantage: the only thing to explain of what you did is : "I typed command X and Y appeared on the screen." (nothing like "I clicked X and then Y and put in Z at A and marked B and then clicked OK and then ....") Your level in using python seems to be the one of a total beginner.

Well here are your commands: open the console run "<path_to_python27_directory>\python.exe <path_to_python27_directory>\Scripts\uncompyle2 --help" replace <path_to_python27_directory> with the path to your own python directory. I do not know where you installed it. In my case it would be "C:\Program Files\Python27"\ (including the "s because of the space char in the path) reason: on linux you can python scripts directly on the console, on windows maybe but I want to be sure thats why this command calls python explicitly to run the uncompyle2 script that should be installed in the subdirectory Scripts of your Python27 installation. If it cannot find python then you put in a wrong path. If it cannot find uncompyle2 then locate uncompyle2 yourself. that will display the commands. Read them. next run <path_to_python27_directory>\python.exe <path_to_python27_directory>\Scripts\uncompyle2 -o <path_of_output_dir> <path> <path_you_never_told>\usr\bin\folder\xyz.pyc

You can probably drop the [...]\python.exe part but that depends on your python installation.

Why are you using suddenly using linux path seperators if you are using windows xp??? Why did you try "usr/bin/folder/xyz.pyc > xyz.py" as command? Did anybody tell you to do this? Are you aware of what the command is actually going to do? If you read it somewhere then where? So far your problem is not using uncompile but how to use python.

EDIT: fixed greater than and less than chars which caused github to remove some textparts from my comment and made the paths look wrong.

[NadhiyaAnbu]

@Ophidiophobia my issue is somewhat related to this topic thats why i asked here

[Ophidiophobia]

I fail to see how. Please create a new issue where everybody can stick to your problem. Your problem IS NOT related to this one here.