Unicodeerror for u266f in Python 3.7 file

markusschloesser commented 1 year ago

Hi,

Description.

I have a file (Midi Remote Script for Ableton, OK to decompile), which has the char u266f (which the musical sign for sharp #). Uncompyle6 will fail for that file with a unicodeerror:

How to Reproduce

import os
import uncompyle6
your_directory = r'M:\Music Software\mackie c4 pro c4c_pc_v1.0\Live 11.3\11.3.1b5\ableton\v3\base'
for dirpath, b, filenames in os.walk(your_directory):
    for filename in filenames:
        if not filename.endswith('.pyc'):
            continue

        filepath = dirpath + '/' + filename
        original_filename = filename.split('.')[0]
        original_filepath = dirpath + '/' + original_filename + '.py'
        with open(original_filepath, 'w') as f:
            uncompyle6.decompile_file(filepath, f)

Python bytecode attached: util.zip

-->

Output Given

Traceback (most recent call last):
  File "C:\Users\Markus\PycharmProjects\C4P3\z_utility\tools MS\decompyle3 test.py", line 13, in <module>
    decompyle3.decompile_file(filepath, f)
  File "C:\Users\Markus\AppData\Roaming\Python\Python310\site-packages\decompyle3\main.py", line 231, in decompile_file
    decompile(
  File "C:\Users\Markus\AppData\Roaming\Python\Python310\site-packages\decompyle3\main.py", line 158, in decompile
    deparsed = deparse_fn(
  File "C:\Users\Markus\AppData\Roaming\Python\Python310\site-packages\decompyle3\semantics\pysource.py", line 1181, in code_deparse
    deparsed.gen_source(
  File "C:\Users\Markus\AppData\Roaming\Python\Python310\site-packages\decompyle3\semantics\pysource.py", line 986, in gen_source
    printfn(self.text)
  File "C:\Users\Markus\AppData\Roaming\Python\Python310\site-packages\decompyle3\semantics\pysource.py", line 482, in println
    self.write(*data)
  File "C:\Users\Markus\AppData\Roaming\Python\Python310\site-packages\decompyle3\semantics\pysource.py", line 478, in write
    self.f.write(out)
  File "C:\Users\Markus\AppData\Local\Programs\Thonny\lib\encodings\cp1252.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_table)[0]
UnicodeEncodeError: 'charmap' codec can't encode character '\u266f' in position 696: character maps to <undefined>

Expected behavior

Not fail over the sharp sign

Environment

Win10 , Thonny, uncomyple6 version 3.9, other files work fine

rocky commented 1 year ago

I am not finding a problem decompiling this (under Ubuntu) with the current master branch of uncompyle and the current master branch of xdis:

$ uncompyle6 util.pyc 
# uncompyle6 version 3.9.1.dev0
# Python bytecode version base 3.7.0 (3394)
# Decompiled from: Python 3.8.16 (default, Dec 13 2022, 19:27:23) 
# [GCC 11.3.0]
# Embedded file name: ..\..\..\output\Live\win_64_static\Release\python-bundle\MIDI Remote Scripts\ableton\v3\base\util.py
# Compiled at: 2023-05-11 03:40:46
# Size of source mod 2**32: 2708 bytes
from __future__ import absolute_import, print_function, unicode_literals
from . import is_iterable
PITCH_NAMES = ('C', 'C#', 'D', 'D#', 'E', 'F', 'F#', 'G', 'G#', 'A', 'A#', 'B')

class CallableBool:

    def __init__(self, value: bool):
        self.value = value

    def __call__(self):
        return self.value

    def __eq__(self, other):
        return self.value == other

    def __bool__(self):
        return self.value

    def __int__(self):
        return int(self.value)

    def __repr__(self):
        return repr(self.value)

def get_default_ascii_translations():
    ascii_translations = {chr(i): i for i in range(32, 127)}
    ascii_translations[u'\u'] = 35
    return ascii_translations

DEFAULT_ASCII_TRANSLATIONS = get_default_ascii_translations()

def as_ascii(string, ascii_translations=DEFAULT_ASCII_TRANSLATIONS):
    result = []
    for char in string:
        translated_char = ascii_translations.get(char, ascii_translations['?'])
        if is_iterable(translated_char):
            result.extend(translated_char)
        else:
            result.append(translated_char)

    return result

def hex_to_rgb(hex_value):
    return (
     (hex_value & 16711680) >> 16,
     (hex_value & 65280) >> 8,
     hex_value & 255)

def pitch_index_to_string(index, pitch_names=PITCH_NAMES):
    return pitch_names[index % 12] + str(index // 12 - 2)
# okay decompiling util.pyc

markusschloesser commented 1 year ago

Thanks! I checked and my xdis is v 6.0.5. Also have another file with the same error physical_display.zip So any idea, why this fails on my Windows machine?

rocky commented 1 year ago

~~Are you using what is in git or what was released? You need to use what is in git.~~

You are not using the version of xdis that is in the master branch. That is version is 6.1.0.dev0

markusschloesser commented 1 year ago

that worked, thank you!

Are you interested in further issues? (Unfortunately I cannot contribute with code)

rocky commented 1 year ago

that worked, thank you!

Are you interested in further issues? (Unfortunately I cannot contribute with code)

Everyone is free to report bugs. The ones that show effort to narrow the problem down and are thoughtful - follow the information given in posting a bug report or reflect that https://github.com/rocky/python-uncompyle6/blob/master/HOW-TO-REPORT-A-BUG.md has been read and followed - are the one that are more likely to get fixed sooner.

But let me answer your question with one of my own: Are you interested in funding the project? See also this where I talk about the frequent comment: "Unfortunately I cannot contribute with code".

rocky / python-uncompyle6