UnicodeDecodeError Handing issue

[stseira]

I've installed Modific to my sublime text but could not show diff on some files. So I turned debug option on, console shows the error below.

Exception in thread Thread-243:
Traceback (most recent call last):
  File ".\threading.py", line 532, in __bootstrap_inner
  File ".\Modific.py", line 121, in run
  File ".\Modific.py", line 71, in _make_text_safeish
  File ".\encodings\cp1252.py", line 15, in decode
UnicodeDecodeError: 'charmap' codec can't decode byte 0x90 in position 574: character maps to <undefined>

I checked out this error happened at source files contained not ascii chars. And I resolved this error changing plugins source code below

def _make_text_safeish(text, fallback_encoding, method='decode'):
    # The unicode decode here is because sublime converts to unicode inside
    # insert in such a way that unknown characters will cause errors, which is
    # distinctly non-ideal... and there's no way to tell what's coming out of
    # git in output. So...
    try:
        unitext = getattr(text, method)('utf-8')
    except (UnicodeEncodeError, UnicodeDecodeError):
        #unitext = getattr(text, method)(fallback_encoding)
        unitext = str(text)
    except AttributeError:
        # strongly implies we're already unicode, but just in case let's cast
        # to string
        unitext = str(text)
    return unitext

but I'm not good at python so I don't know this change will cause other side effect. plz check out this issue.

[stseira]

I found other resolution.

    except (UnicodeEncodeError, UnicodeDecodeError):
        if 'Windows 1252' == fallback_encoding:
            fallback_encoding = 'ISO 8859-1'
        unitext = getattr(text, method)(fallback_encoding)

[gornostal]

Hi @stseira,

If this is still the issue for you, could you please share a file that I can use for testing? Before pushing any changes to master I want to make sure the problem exists, and your solution works on all platforms.

Thanks.

[stseira]

Oh sorry.. I'm too late to check out your comment. I have reinstalled sublime text and modific a few days ago... (the plugin codes not changed.) but that is not issue.

anyway.. the file have issued is this one..

<?php echo 'foo'; //the file including 한국어 문자 <- that is korean alphabet

[fhemberger]

Thanks @stseira, your second code example fixed this issue for me on OS X Mavericks (was fine on 10.8 before).

[gornostal]

@stseira can try to reproduce it again? I just pushed fix that hopefully will fix your issue too.

[gornostal]

Please reopen if needed.