Add errors='replace' to avoid UTF-8 decoding error

[guosasipo]

Some PCs get UTF-8 decoding error so it needs errors='replace' option to avoid that

[fritzlb]

Thank you for the suggestion! Do you have any examples (on languages/other setups/...) that cause this issue? I'd rather not fix hypothetical issues that don't ever happen.

[guosasipo]

I use korean version of windows 11 and it makes error like this

C:\IOS17-~1> python jit_enabler_better.py com.utmapp.UTM.QFH2Z83KZW Getting bundle ID... Got bundle ID: com.utmapp.UTM.QFH2Z83KZW starting tunnel to device... This might take a while. In case it freezes, either close this window and kill every python process in task manager or simply reboot your PC. Sucessfully created tunnel: fdc9:69c9:10b3::1 63150 Manually trying to mount DeveloperDiskImage (this seems to prevent errors on some systems)... Traceback (most recent call last): File "C:\IOS17-~1\jit_enabler_better.py", line 70, in ret_val = dev_img_proc.communicate()[1].decode() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc0 in position 34: invalid start byte

[fritzlb]

Did you investigate on what characters are returned from the dev_img_proc that can't be decoded? If I understand it correctly, encoding='replace' "fixes" it by ignoring/replacing all undecodable characters. This isn't the cleanest method but probably fine, so I'll just merge it anyway. In case we find out what characters are causing it we can always improve the fix. Thank you so much for finding and fixing this!

[guosasipo]

well it's not only issue with dev_img_proc. It also makes error with pid which is just number

[fritzlb]

I kinda suspected this... once characters are replaced a comparison between a predefined string and something definitely containing characters not in that string is false...

Guess I'll revert and try to come up with a better idea

[guosasipo]

I checked raw data about 'dev_img_proc' and it printed this value which utf-8 can't decode

b'\x1b[32m2024-04-19 17:53:32\x1b[0m \x1b[35m\xc0\xcc\xc8\xa3\xc1\xd8\x1b[0m \x1b[34mpymobiledevice3.cli.mounter[28652]\x1b[0m \x1b[1;30mERROR\x1b[0m \x1b[31mDeveloperDiskImage already mounted\x1b[0m\r\n'

[fritzlb]

Very interesting, on my system it's b'\x1b[32m2024-04-19 11:44:13\x1b[0m \x1b[35mFrederiksPC\x1b[0m \x1b[34mpymobiledevice3.cli.mounter[7268]\x1b[0m \x1b[1;30mERROR\x1b[0m \x1b[31mDeveloperDiskImage already mounted\x1b[0m\r\n' Any chance your PC name has special characters in it?

[guosasipo]

ah maybe my pc name has korean in it so it makes error

[fritzlb]

Got it. ks_c_5601-1987 is the encoding that has to be used here it seems. For you, adding encoding='ks_c_5601-198' should fix it. I'll test around a little, maybe detect if the user has korean as language set and then use that or sth.

[guosasipo]

hmm if I add encoding='ks_c_5601-1987' it will make error on python lib like below since python use cp949 codec to decode korean

python jit_enabler_better.py com.utmapp.UTM.QFH2Z83KZW Getting bundle ID... Got bundle ID: com.utmapp.UTM.QFH2Z83KZW starting tunnel to device... This might take a while. In case it freezes, either close this window and kill every python process in task manager or simply reboot your PC. Sucessfully created tunnel: fd12:5c3e:6f49::1 63170 Manually trying to mount DeveloperDiskImage (this seems to prevent errors on some systems)... Diskimage already mounted. Starting app... Started app. PID: 2518 Starting debug server... Started debug server with connection details: [fd12:5c3e:6f49::1]:53279 Run debugging commands... This might take a few minutes. Exception in thread Thread-5 (_readerthread): Traceback (most recent call last): File "C:\Program Files\Python312\Lib\threading.py", line 1073, in _bootstrap_inner self.run() File "C:\Program Files\Python312\Lib\threading.py", line 1010, in run self._target(*self._args, **self._kwargs) File "C:\Program Files\Python312\Lib\subprocess.py", line 1597, in _readerthread buffer.append(fh.read()) ^^^^^^^^^ UnicodeDecodeError: 'cp949' codec can't decode byte 0xed in position 83: illegal multibyte sequence done.

[fritzlb]

Oh, then probably the output from lldb is utf-8 encoded. What a mess… Does it work if you remove the encoding from everything after the debugging commands? I should probably use the pymobiledevice3 API instead of whatever shell code mess I produced. Well, too late now. Also by now there’s a new sideJIT compatible with iOS 17, so I don’t think rewriting everything would be worth it