Python 3.8 "breaks" are broken

[treatmesubj]

Description

Expected while true try break except loop. I believe the Python is functionally fine, but I may be stupid, I'm just playing around and experimenting. Looks like the bytecode is right.

How to Reproduce

Used PyInstaller to create an executable with this dummy script, which was the portion of code that seemed to be causing the parsing error in my actual script. Used pyinstxtractor.py to separate out .pyc files from .exe. Added the magic bytes and timestamp at the head of .pyc file. Used uncompyle6 on test.pyc and parsing error was raised. I also tried decompyle3, but found the same error.

Expected Python Script

import traceback
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
import os

try:

    def checkbox(CSSselector, boolie):
        while True:
            try:
                WebDriverWait(driver, 10).until(EC.element_to_be_clickable(
                    (By.CSS_SELECTOR, CSSselector))).click()
                if driver.find_element_by_css_selector(CSSselector).is_selected() is boolie:
                    print(CSSselector + ' check status good')
                    break
            except Exception:
                pass

    driver = webdriver.Firefox(service_log_path=os.path.devnull)
    driver.maximize_window()

    while True:
        try:
            print('clicking print setup button...')
            WebDriverWait(driver, 10).until(EC.element_to_be_clickable(
                (By.CSS_SELECTOR, '#printSetupBttn'))).click()
            print('clicked...')
            driver.find_element_by_css_selector('#printSetupOKBtn')
            print('setup menu appears to be up')
            break
        except Exception:
            pass
    checkbox('#downloadRpt', False)
    checkbox('#printLocalRpt', True)

except Exception:
    traceback.print_exc()

$ uncompyle6 test.pyc

Bytecode

550d 0d0a 0100 0000 0000 0000 0000 0000
e300 0000 0000 0000 0000 0000 0000 0000
0008 0000 0040 0000 0073 fc00 0000 6400
6401 6c00 5a00 6400 6402 6c01 6d02 5a02
0100 6400 6403 6c03 6d04 5a04 0100 6400
6404 6c05 6d06 5a07 0100 6400 6405 6c08
6d09 5a09 0100 6400 6401 6c0a 5a0a 7a9a
6406 6407 8400 5a0b 6502 6a0c 650a 6a0d
6a0e 6408 8d01 5a0f 650f a010 a100 0100
7a4a 6511 6409 8301 0100 6504 650f 640a
8302 a012 6507 a013 6509 6a14 640b 6602
a101 a101 a015 a100 0100 6511 640c 8301
0100 650f a016 640d a101 0100 6511 640e
8301 0100 5700 71c4 5700 7162 0400 6517
6b0a 72c0 0100 0100 0100 5900 7162 5800
7162 650b 640f 6410 8302 0100 650b 6411
6412 8302 0100 5700 6e1c 0400 6517 6b0a
72f6 0100 0100 0100 6500 a018 a100 0100
5900 6e02 5800 6401 5300 2913 e900 0000
004e 2901 da09 7765 6264 7269 7665 7229
01da 0d57 6562 4472 6976 6572 5761 6974
2901 da13 6578 7065 6374 6564 5f63 6f6e
6469 7469 6f6e 7329 01da 0242 7963 0200
0000 0000 0000 0000 0000 0200 0000 0800
0000 4300 0000 7362 0000 007a 4674 0074
0164 0183 02a0 0274 03a0 0474 056a 067c
0066 02a1 01a1 01a0 07a1 0001 0074 01a0
087c 00a1 01a0 09a1 007c 016b 0872 4474
0a7c 0064 0217 0083 0101 0057 0071 5e57
0071 0004 0074 0b6b 0a72 5a01 0001 0001
0059 0071 0058 0071 0064 0053 0029 034e
e90a 0000 007a 1220 6368 6563 6b20 7374
6174 7573 2067 6f6f 6429 0c72 0300 0000
da06 6472 6976 6572 da05 756e 7469 6cda
0245 43da 1765 6c65 6d65 6e74 5f74 6f5f
6265 5f63 6c69 636b 6162 6c65 7205 0000
00da 0c43 5353 5f53 454c 4543 544f 52da
0563 6c69 636b da1c 6669 6e64 5f65 6c65
6d65 6e74 5f62 795f 6373 735f 7365 6c65
6374 6f72 da0b 6973 5f73 656c 6563 7465
64da 0570 7269 6e74 da09 4578 6365 7074
696f 6e29 02da 0b43 5353 7365 6c65 6374
6f72 da06 626f 6f6c 6965 a900 7213 0000
00fa 0774 6573 742e 7079 da08 6368 6563
6b62 6f78 0a00 0000 7312 0000 0000 0202
010e 0108 ff0a 0212 010c 0108 010e 0172
1500 0000 2901 da10 7365 7276 6963 655f
6c6f 675f 7061 7468 7a1e 636c 6963 6b69
6e67 2070 7269 6e74 2073 6574 7570 2062
7574 746f 6e2e 2e2e 7206 0000 007a 0f23
7072 696e 7453 6574 7570 4274 746e 7a0a
636c 6963 6b65 642e 2e2e 7a10 2370 7269
6e74 5365 7475 704f 4b42 746e 7a1b 7365
7475 7020 6d65 6e75 2061 7070 6561 7273
2074 6f20 6265 2075 707a 0c23 646f 776e
6c6f 6164 5270 7446 7a0e 2370 7269 6e74
4c6f 6361 6c52 7074 5429 19da 0974 7261
6365 6261 636b da08 7365 6c65 6e69 756d
7202 0000 00da 1d73 656c 656e 6975 6d2e
7765 6264 7269 7665 722e 7375 7070 6f72
742e 7569 7203 0000 00da 1a73 656c 656e
6975 6d2e 7765 6264 7269 7665 722e 7375
7070 6f72 7472 0400 0000 7209 0000 00da
1c73 656c 656e 6975 6d2e 7765 6264 7269
7665 722e 636f 6d6d 6f6e 2e62 7972 0500
0000 da02 6f73 7215 0000 00da 0746 6972
6566 6f78 da04 7061 7468 da07 6465 766e
756c 6c72 0700 0000 da0f 6d61 7869 6d69
7a65 5f77 696e 646f 7772 0f00 0000 7208
0000 0072 0a00 0000 720b 0000 0072 0c00
0000 720d 0000 0072 1000 0000 da09 7072
696e 745f 6578 6372 1300 0000 7213 0000
0072 1300 0000 7214 0000 00da 083c 6d6f
6475 6c65 3e01 0000 0073 3000 0000 0801
0c01 0c01 0c01 0c01 0802 0202 080b 1001
0803 0201 0801 0e01 08ff 0a02 0801 0a01
0801 0801 0e01 0801 0a01 0e02 0e01

Console Output

C:\Users\JohnHupperts\Desktop\Python Projects\CostpointAutomation\testdecompiles\dist\test\test.exe_extracted>uncompyle6 test.pyc
# uncompyle6 version 3.5.1
# Python bytecode 3.8 (3413)
# Decompiled from: Python 3.8.0 (tags/v3.8.0:fa919fd, Oct 14 2019, 19:21:23) [MSC v.1916 32 bit (Intel)]
# Embedded file name: test.py
Instruction context:

  32     166  POP_BLOCK
->          168  JUMP_ABSOLUTE       196  'to 196'
            170  POP_BLOCK
            172  JUMP_BACK            98  'to 98'
          174_0  COME_FROM_FINALLY    98  '98'

# file test.pyc
# --- This code section failed: ---

   1       0  LOAD_CONST               0
           2  LOAD_CONST               None
           4  IMPORT_NAME              traceback
           6  STORE_NAME               traceback

   2       8  LOAD_CONST               0
          10  LOAD_CONST               ('webdriver',)
          12  IMPORT_NAME              selenium
          14  IMPORT_FROM              webdriver
          16  STORE_NAME               webdriver
          18  POP_TOP

   3      20  LOAD_CONST               0
          22  LOAD_CONST               ('WebDriverWait',)
          24  IMPORT_NAME              selenium.webdriver.support.ui
          26  IMPORT_FROM              WebDriverWait
          28  STORE_NAME               WebDriverWait
          30  POP_TOP

   4      32  LOAD_CONST               0
          34  LOAD_CONST               ('expected_conditions',)
          36  IMPORT_NAME              selenium.webdriver.support
          38  IMPORT_FROM              expected_conditions
          40  STORE_NAME               EC
          42  POP_TOP

   5      44  LOAD_CONST               0
          46  LOAD_CONST               ('By',)
          48  IMPORT_NAME              selenium.webdriver.common.by
          50  IMPORT_FROM              By
          52  STORE_NAME               By
          54  POP_TOP

   6      56  LOAD_CONST               0
          58  LOAD_CONST               None
          60  IMPORT_NAME              os
          62  STORE_NAME               os

   8      64  SETUP_FINALLY       220  'to 220'

  10      66  LOAD_CODE                <code_object checkbox>
          68  LOAD_STR                 'checkbox'
          70  MAKE_FUNCTION_0          ''
          72  STORE_NAME               checkbox

  21      74  LOAD_NAME                webdriver
          76  LOAD_ATTR                Firefox
          78  LOAD_NAME                os
          80  LOAD_ATTR                path
          82  LOAD_ATTR                devnull
          84  LOAD_CONST               ('service_log_path',)
          86  CALL_FUNCTION_KW_1     1  ''
          88  STORE_NAME               driver

  22      90  LOAD_NAME                driver
          92  LOAD_METHOD              maximize_window
          94  CALL_METHOD_0         0  ''
          96  POP_TOP

  25      98  SETUP_FINALLY       174  'to 174'

  26     100  LOAD_NAME                print
         102  LOAD_STR                 'clicking print setup button...'
         104  CALL_FUNCTION_1       1  ''
         106  POP_TOP

  27     108  LOAD_NAME                WebDriverWait
         110  LOAD_NAME                driver
         112  LOAD_CONST               10
         114  CALL_FUNCTION_2       2  ''
         116  LOAD_METHOD              until
         118  LOAD_NAME                EC
         120  LOAD_METHOD              element_to_be_clickable

  28     122  LOAD_NAME                By
         124  LOAD_ATTR                CSS_SELECTOR
         126  LOAD_STR                 '#printSetupBttn'
         128  BUILD_TUPLE_2         2

  27     130  CALL_METHOD_1         1  ''
         132  CALL_METHOD_1         1  ''
         134  LOAD_METHOD              click
         136  CALL_METHOD_0         0  ''
         138  POP_TOP

  29     140  LOAD_NAME                print
         142  LOAD_STR                 'clicked...'
         144  CALL_FUNCTION_1       1  ''
         146  POP_TOP

  30     148  LOAD_NAME                driver
         150  LOAD_METHOD              find_element_by_css_selector
         152  LOAD_STR                 '#printSetupOKBtn'
         154  CALL_METHOD_1         1  ''
         156  POP_TOP

  31     158  LOAD_NAME                print
         160  LOAD_STR                 'setup menu appears to be up'
         162  CALL_FUNCTION_1       1  ''
         164  POP_TOP

  32     166  POP_BLOCK
         168  JUMP_ABSOLUTE       196  'to 196'
         170  POP_BLOCK
         172  JUMP_BACK            98  'to 98'
       174_0  COME_FROM_FINALLY    98  '98'

  33     174  DUP_TOP
         176  LOAD_NAME                Exception
         178  COMPARE_OP               exception-match
         180  POP_JUMP_IF_FALSE   192  'to 192'
         182  POP_TOP
         184  POP_TOP
         186  POP_TOP

  34     188  POP_EXCEPT
         190  JUMP_BACK            98  'to 98'
         192  END_FINALLY
         194  JUMP_BACK            98  'to 98'

  35     196  LOAD_NAME                checkbox
         198  LOAD_STR                 '#downloadRpt'
         200  LOAD_CONST               False
         202  CALL_FUNCTION_2       2  ''
         204  POP_TOP

  36     206  LOAD_NAME                checkbox
         208  LOAD_STR                 '#printLocalRpt'
         210  LOAD_CONST               True
         212  CALL_FUNCTION_2       2  ''
         214  POP_TOP
         216  POP_BLOCK
         218  JUMP_FORWARD        248  'to 248'
       220_0  COME_FROM_FINALLY    64  '64'

  38     220  DUP_TOP
         222  LOAD_NAME                Exception
         224  COMPARE_OP               exception-match
         226  POP_JUMP_IF_FALSE   246  'to 246'
         228  POP_TOP
         230  POP_TOP
         232  POP_TOP

  39     234  LOAD_NAME                traceback
         236  LOAD_METHOD              print_exc
         238  CALL_METHOD_0         0  ''
         240  POP_TOP
         242  POP_EXCEPT
         244  JUMP_FORWARD        248  'to 248'
         246  END_FINALLY
       248_0  COME_FROM           244  '244'
       248_1  COME_FROM           218  '218'

Parse error at or near `JUMP_ABSOLUTE' instruction at offset 168

Environment

Windows 10, Python 3.8, uncompyle6 - I pip installed about an hour ago

[rocky]

In the "honest-come-froms" branch of decompiyle3 this is fixed by this commit. That branch is where I'm trying to address some of the long-standing and pervasive control-flow problems in this approach. And I am afraid that's going to take a while. (But were it done this code would be much more solid)

The commit cited could probably be backported here, but I want someone else to handle that. Are you up for it? Or maybe @x0ret ?

As for the specific problem itself. Python 3.8 removed BREAK_LOOP and CONTINUE_LOOP opcodes. in the case of BREAK_LOOP that is replaced by a POP_BLOCK, JUMP_ABSOLUTE around offset 168 shown in the error above.

In general I expect continue and break to be problematic in Python 3.8.

Fiinally, let me say that I don't think decompiling 3.8 is all that great. It's just the best thing out there right now. (And I thiink it would be hard to write something better from scratch unless it borrows from the code here.)

[treatmesubj]

For sure way beyond anything I would understand, but I'll definitely take a look out of interest ha. decompile3 honest-come-froms with the new commit definitely fixed that particular break. There probably can't be many 3.8 .pycs yet. PyInstaller master branch doesn't support 3.8 yet either.

[rocky]

At some point I backported this from decompyle3