Full unicode compatibility

hakril commented 5 months ago

The goal of this PR is to have full unicode compatibility by using W api whenever possible. This results of the use of unicode string in python2 & str in py3.

This will also allow to handle process, filenames, certificates & co with unicode name.

This will be merged when no more A API are used where W exists and when code is tested and pass on both py2 & py3.

Still TODO:

[x] event_trace.py
[x] winutils.py
[x] system.py
[x] volume.py
[x] Dbghelp.py
[x] device_manager.py

github-actions[bot] commented 5 months ago

Unit Test Results

433 tests +5 425 :heavy_check_mark: +36 2m 12s :stopwatch: -32s     1 suites ±0     7 :zzz: - 32     1 files ±0     1 :x: +  1

For more details on these failures, see this check.

Results for commit 529e3a58. ± Comparison against base commit 275d054b.

This pull request removes 50 and adds 55 tests. Note that renamed tests count towards both.

``` test_ndr ‑ test_ndr_packing[ComplexAlignementStructure-values12-APPP\x02\x02\x02\x02\x03\x03\x03\x03DPPP\x05\x05\x05\x05\x06\x06\x06\x06BPPPCCCCEEEEEEEEF] test_ndr ‑ test_ndr_packing[NdrContextHandle-values5-\x00\x00\x00\x00D3"\x11"\x11D3UfBCDEFG] test_ndr ‑ test_ndr_packing[NdrCustomParameters-values10-\x01\x01PPAABPCCCCDPEEFG] test_ndr ‑ test_ndr_packing[NdrCustomParameters-values11-\x01\x01PPAABPCCCCDPEEFPGG] test_ndr ‑ test_ndr_packing[NdrCustomParameters-values3-\x01\x01\x02] test_ndr ‑ test_ndr_packing[NdrCustomParameters-values4-\x01PPPPPPPAAAAAAAABPPPCCCCDDDDDDDD] test_ndr ‑ test_ndr_packing[NdrCustomParameters-values7-\x01\x01PPAAAABBBBCCCCDDDDEEEE] test_ndr ‑ test_ndr_packing[NdrCustomParameters-values8-\x01\x01PPAAAABBBBCCCCDDDDEEEE] test_ndr ‑ test_ndr_packing[NdrCustomStructure-values1-\x01\x01\x02] test_ndr ‑ test_ndr_packing[NdrCustomStructure-values2-\x01\x01\x02\x02] … ``` ``` test_crypto ‑ test_unicode_certificate test_ndr ‑ test_ndr_packing[ComplexAlignementStructure-values15-APPP\x02\x02\x02\x02\x03\x03\x03\x03DPPP\x05\x05\x05\x05\x06\x06\x06\x06BPPPCCCCEEEEEEEEF] test_ndr ‑ test_ndr_packing[NdrContextHandle-values2-\x00\x00\x00\x00BBBBBBBBBBBBBBBB] test_ndr ‑ test_ndr_packing[NdrContextHandle-values7-\x00\x00\x00\x00D3"\x11"\x11D3UfBCDEFG] test_ndr ‑ test_ndr_packing[NdrCustomParameters-values10-\x01\x01PPAAAABBBBCCCCDDDDEEEE] test_ndr ‑ test_ndr_packing[NdrCustomParameters-values12-BBBB\x04\x00\x00\x00\x00\x00\x00\x00\x04\x00\x00\x00ABCDCCCCDDDD] test_ndr ‑ test_ndr_packing[NdrCustomParameters-values13-\x01\x01PPAABPCCCCDPEEFG] test_ndr ‑ test_ndr_packing[NdrCustomParameters-values14-\x01\x01PPAABPCCCCDPEEFPGG] test_ndr ‑ test_ndr_packing[NdrCustomParameters-values5-\x01\x01\x02] test_ndr ‑ test_ndr_packing[NdrCustomParameters-values6-\x01PPPPPPPAAAAAAAABPPPCCCCDDDDDDDD] … ```

This pull request skips 6 and un-skips 38 tests.

``` test_native_utils.TestNativeUtils ‑ test_getprocaddr32 test_native_utils.TestNativeUtils ‑ test_strlena32 test_native_utils.TestNativeUtils ‑ test_strlenw32 test_syswow ‑ test_syswow_call_multithread test_syswow.TestSyswowCurrentProcess ‑ test_exec_syswow test_syswow.TestSyswowCurrentProcess ‑ test_self_pebsyswow ``` ``` test_debugger ‑ test_bp_exe_by_name[proc64dbg] test_debugger ‑ test_exe_in_module_list[proc64dbg] test_debugger ‑ test_four_hwx_breakpoint_fail[proc64dbg] test_debugger ‑ test_hardware_breakpoint_name_addr[proc64dbg] test_debugger ‑ test_hwx_breakpoint_are_on_all_thread[proc64dbg] test_debugger ‑ test_init_breakpoint_callback[proc64dbg] test_debugger ‑ test_keyboardinterrupt_when_bp_event[proc64dbg] test_debugger ‑ test_mem_breakpoint_remove[proc64dbg] test_debugger ‑ test_mem_breakpoint_self_remove[proc64dbg] test_debugger ‑ test_memory_breakpoint_exec[proc64dbg] … ```