Inefficient process enumeration on Windows

[Still34]

Summary

Earlier this month, I was troubleshooting the enormous WMI host CPU usage on my system (consistently around ~5% CPU usage). Through WMIMon, I found that the culprit was the Profile Autoswitcher app. The current implementation appears to query a WMI call to the system every 0.25 second if I'm reading this correctly.

https://github.com/dekuNukem/duckyPad-profile-autoswitcher/blob/b9f690a459630a018ec1bd46407e958b3668b0f4/src/duckypad_autoprofile.py#L640

https://github.com/dekuNukem/duckyPad-profile-autoswitcher/blob/b9f690a459630a018ec1bd46407e958b3668b0f4/src/get_window.py#L109

This constant loop of creating WMI queries causes a heavy strain on the WMI host process.

Suggested Fix

Fortunately, there are other methods of querying for process name with PID. One of the methods is to use native Win32 API instead of having to have a roundtrip to the WMI. Using NtQuerySystemInformation with the SystemProcessIdInformation struct, we can get the PID and the process name without any of the surplus.

The following is a PoC that demonstrates how this might be done in Python via ctwin32, a tiny Win32 abstraction library for Python (though this can be done with just the native Python ctypes I believe).

import ctwin32
import ctwin32.ntdll
import ctypes
import ctwin32.wtypes
import wmi
import cProfile

def get_process_name_native(pid: int):
    spii = ctwin32.ntdll.SYSTEM_PROCESS_ID_INFORMATION()
    buffer = ctypes.create_unicode_buffer(0x1000)
    spii.ProcessId = pid
    spii.ImageName.MaximumLength = len(buffer)
    spii.ImageName.Buffer = ctypes.addressof(buffer)
    status = ctwin32.ntdll.NtQuerySystemInformation(ctwin32.SystemProcessIdInformation,ctypes.byref(spii),ctypes.sizeof(spii), None)
    name = str(spii.ImageName)
    dot = name.rfind('.')
    slash = name.rfind('\\')
    return name[slash+1:dot]

def get_process_name_wmi(pid: int):
    c = wmi.WMI()
    for p in c.query('SELECT Name FROM Win32_Process WHERE ProcessId = %s' % str(pid)):
        exe = p.Name.rsplit('.', 1)[0]
    return exe

cProfile.run('get_process_name_native(38292)')
cProfile.run('get_process_name_wmi(38292)')

The profiling shows a huge improvement in overall execution time and there's no stress being done to the WMI process.

         17 function calls in 0.000 seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000    0.000    0.000 <string>:1(<module>)
        1    0.000    0.000    0.000    0.000 __init__.py:273(create_unicode_buffer)
        1    0.000    0.000    0.000    0.000 __init__.py:525(wstring_at)
        1    0.000    0.000    0.000    0.000 ntdll.py:262(NtQuerySystemInformation)
        1    0.000    0.000    0.000    0.000 poc.py:8(get_process_name_native)
        1    0.000    0.000    0.000    0.000 wtypes.py:264(__str__)
        1    0.000    0.000    0.000    0.000 {built-in method _ctypes.addressof}
        1    0.000    0.000    0.000    0.000 {built-in method _ctypes.byref}
        1    0.000    0.000    0.000    0.000 {built-in method _ctypes.sizeof}
        1    0.000    0.000    0.000    0.000 {built-in method builtins.exec}
        2    0.000    0.000    0.000    0.000 {built-in method builtins.isinstance}
        1    0.000    0.000    0.000    0.000 {built-in method builtins.len}
        1    0.000    0.000    0.000    0.000 {built-in method sys.audit}
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
        2    0.000    0.000    0.000    0.000 {method 'rfind' of 'str' objects}

         2475 function calls (2459 primitive calls) in 0.237 seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000    0.000    0.000 <COMObject winmgmts:>:1(<module>)
        1    0.000    0.000    0.002    0.002 <COMObject winmgmts:>:1(ExecQuery)
        1    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:100(acquire)
        1    0.000    0.000    0.001    0.001 <frozen importlib._bootstrap>:1022(_find_and_load)
    30/29    0.000    0.000    0.001    0.000 <frozen importlib._bootstrap>:1053(_handle_fromlist)
        1    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:125(release)
        1    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:165(__init__)
        1    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:169(__enter__)
        1    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:173(__exit__)
        1    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:179(_get_module_lock)
        1    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:198(cb)
      2/1    0.000    0.000    0.001    0.001 <frozen importlib._bootstrap>:233(_call_with_frames_removed)
        6    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:244(_verbose_message)
        1    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:357(__init__)
        2    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:391(cached)
       30    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:404(parent)
        1    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:412(has_location)
        1    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:48(_new_module)
        1    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:492(_init_module_attrs)
        1    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:564(module_from_spec)
        1    0.000    0.000    0.001    0.001 <frozen importlib._bootstrap>:664(_load_unlocked)
        1    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:71(__init__)
        1    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:746(find_spec)
        1    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:826(find_spec)
        5    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:893(__enter__)
        5    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:897(__exit__)
        1    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:921(_find_spec)
        1    0.000    0.000    0.001    0.001 <frozen importlib._bootstrap>:987(_find_and_load_unlocked)
        1    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:1040(__init__)
        1    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:1065(get_filename)
        1    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:1070(get_data)
        1    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:1089(path_stats)
        5    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:119(<listcomp>)
        2    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:132(_path_split)
        6    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:134(<genexpr>)
        1    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:1356(_path_importer_cache)
        1    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:1399(_get_spec)
        3    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:140(_path_stat)
        1    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:1431(find_spec)
        1    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:150(_path_is_mode_type)
        1    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:1531(_get_spec)
        1    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:1536(find_spec)
        1    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:159(_path_isfile)
        1    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:172(_path_isabs)
        2    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:380(cache_from_source)
        1    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:510(_get_cached)
        1    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:542(_check_name_wrapper)
        1    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:585(_classify_pyc)
        1    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:618(_validate_timestamp_pyc)
        1    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:67(_relax_case)
        1    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:670(_compile_bytecode)
        1    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:721(spec_from_file_location)
        3    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:84(_unpack_uint32)
        1    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:874(create_module)
        1    0.000    0.000    0.001    0.001 <frozen importlib._bootstrap_external>:877(exec_module)
        1    0.000    0.000    0.001    0.001 <frozen importlib._bootstrap_external>:950(get_code)
        5    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:96(_path_join)
        1    0.000    0.000    0.237    0.237 <string>:1(<module>)
       78    0.000    0.000    0.000    0.000 <string>:12(__init__)
       24    0.000    0.000    0.000    0.000 CLSIDToClass.py:52(HasClass)
       23    0.000    0.000    0.004    0.000 __init__.py:108(Dispatch)
        8    0.000    0.000    0.002    0.000 __init__.py:160(_wrap_dispatch_)
       24    0.000    0.000    0.005    0.000 __init__.py:19(__WrapDispatch)
        1    0.000    0.000    0.002    0.002 __init__.py:57(GetObject)
       15    0.000    0.000    0.003    0.000 __init__.py:607(_get_good_single_object_)
       15    0.000    0.000    0.003    0.000 __init__.py:613(_get_good_object_)
        1    0.000    0.000    0.000    0.000 __init__.py:89(find_spec)
        1    0.000    0.000    0.002    0.002 __init__.py:99(Moniker)
        1    0.000    0.000    0.000    0.000 build.py:115(GetResultCLSID)
       24    0.000    0.000    0.000    0.000 build.py:139(__init__)
       24    0.000    0.000    0.000    0.000 build.py:155(__init__)
        7    0.000    0.000    0.000    0.000 build.py:167(_propMapPutCheck_)
       27    0.000    0.000    0.000    0.000 build.py:184(_propMapGetCheck_)
       35    0.000    0.000    0.000    0.000 build.py:201(_AddFunc_)
       34    0.000    0.000    0.000    0.000 build.py:333(CountInOutOptArgs)
        1    0.000    0.000    0.000    0.000 build.py:350(MakeFuncMethod)
        1    0.000    0.000    0.000    0.000 build.py:357(MakeDispatchFuncMethod)
        1    0.000    0.000    0.000    0.000 build.py:401(<listcomp>)
        1    0.000    0.000    0.000    0.000 build.py:406(<listcomp>)
        1    0.000    0.000    0.000    0.000 build.py:407(<listcomp>)
       24    0.000    0.000    0.000    0.000 build.py:534(__init__)
    58/46    0.000    0.000    0.000    0.000 build.py:546(_ResolveType)
        1    0.000    0.000    0.000    0.000 build.py:616(_BuildArgList)
        9    0.000    0.000    0.000    0.000 build.py:652(MakePublicAttributeName)
        9    0.000    0.000    0.000    0.000 build.py:682(<listcomp>)
        4    0.000    0.000    0.000    0.000 build.py:690(MakeDefaultArgRepr)
        1    0.000    0.000    0.000    0.000 build.py:721(BuildCallList)
       35    0.000    0.000    0.000    0.000 build.py:83(__init__)
       28    0.000    0.000    0.000    0.000 dynamic.py:107(_GetDescInvokeType)
       24    0.000    0.000    0.000    0.000 dynamic.py:123(Dispatch)
       24    0.000    0.000    0.000    0.000 dynamic.py:152(MakeOleRepr)
       24    0.000    0.000    0.000    0.000 dynamic.py:195(__init__)
        1    0.000    0.000    0.000    0.000 dynamic.py:210(__call__)
        5    0.000    0.000    0.001    0.000 dynamic.py:303(_NewEnum)
       20    0.000    0.000    0.228    0.011 dynamic.py:315(__getitem__)
        1    0.000    0.000    0.002    0.002 dynamic.py:365(_ApplyTypes_)
       45    0.000    0.000    0.002    0.000 dynamic.py:378(_get_good_single_object_)
       45    0.000    0.000    0.002    0.000 dynamic.py:391(_get_good_object_)
        1    0.000    0.000    0.000    0.000 dynamic.py:409(_make_method_)
       29    0.000    0.000    0.002    0.000 dynamic.py:482(__LazyMap__)
       29    0.000    0.000    0.002    0.000 dynamic.py:493(_LazyAddAttr_)
        1    0.000    0.000    0.000    0.000 dynamic.py:544(__AttrToID__)
       30    0.000    0.000    0.004    0.000 dynamic.py:551(__getattr__)
      113    0.000    0.000    0.000    0.000 dynamic.py:59(debug_attr_print)
        1    0.000    0.000    0.000    0.000 dynamic.py:66(MakeMethod)
       47    0.000    0.000    0.000    0.000 dynamic.py:78(_GetGoodDispatch)
       47    0.000    0.000    0.000    0.000 dynamic.py:95(_GetGoodDispatchAndUserName)
       24    0.000    0.000    0.000    0.000 gencache.py:186(GetClassForCLSID)
       24    0.000    0.000    0.000    0.000 gencache.py:227(GetModuleForCLSID)
        1    0.000    0.000    0.000    0.000 importer.py:246(find_spec)
        1    0.000    0.000    0.237    0.237 poc.py:20(get_process_name_wmi)
        1    0.000    0.000    0.000    0.000 util.py:1(<module>)
        5    0.000    0.000    0.000    0.000 util.py:12(WrapEnum)
        1    0.000    0.000    0.000    0.000 util.py:24(Enumerator)
        5    0.000    0.000    0.000    0.000 util.py:36(__init__)
       20    0.000    0.000    0.226    0.011 util.py:40(__getitem__)
       20    0.000    0.000    0.226    0.011 util.py:46(__GetIndex)
        1    0.000    0.000    0.000    0.000 util.py:84(EnumVARIANT)
        5    0.000    0.000    0.000    0.000 util.py:85(__init__)
       15    0.000    0.000    0.003    0.000 util.py:89(_make_retval_)
        1    0.000    0.000    0.000    0.000 util.py:93(Iterator)
        1    0.000    0.000    0.002    0.002 wmi.py:1055(_raw_query)
        1    0.000    0.000    0.233    0.233 wmi.py:1068(query)
        1    0.000    0.000    0.231    0.231 wmi.py:1072(<listcomp>)
        1    0.000    0.000    0.003    0.003 wmi.py:1262(connect)
        1    0.000    0.000    0.000    0.000 wmi.py:1361(construct_moniker)
        1    0.000    0.000    0.001    0.001 wmi.py:1390(get_wmi_type)
       14    0.000    0.000    0.000    0.000 wmi.py:343(_set)
        1    0.000    0.000    0.001    0.001 wmi.py:480(__init__)
        2    0.000    0.000    0.000    0.000 wmi.py:484(<genexpr>)
        1    0.000    0.000    0.006    0.006 wmi.py:519(__init__)
        5    0.000    0.000    0.001    0.000 wmi.py:542(<genexpr>)
        1    0.000    0.000    0.001    0.001 wmi.py:569(_cached_properties)
        1    0.000    0.000    0.001    0.001 wmi.py:579(__getattr__)
        1    0.000    0.000    0.000    0.000 wmi.py:589(<lambda>)
        1    0.000    0.000    0.000    0.000 wmi.py:982(__init__)
        1    0.000    0.000    0.000    0.000 {built-in method _imp._fix_co_filename}
        7    0.000    0.000    0.000    0.000 {built-in method _imp.acquire_lock}
        1    0.000    0.000    0.000    0.000 {built-in method _imp.is_frozen}
        7    0.000    0.000    0.000    0.000 {built-in method _imp.release_lock}
        2    0.000    0.000    0.000    0.000 {built-in method _thread.allocate_lock}
        2    0.000    0.000    0.000    0.000 {built-in method _thread.get_ident}
        3    0.000    0.000    0.000    0.000 {built-in method builtins.__build_class__}
        1    0.000    0.000    0.001    0.001 {built-in method builtins.__import__}
        1    0.000    0.000    0.000    0.000 {built-in method builtins.compile}
      3/1    0.000    0.000    0.237    0.237 {built-in method builtins.exec}
        6    0.000    0.000    0.000    0.000 {built-in method builtins.getattr}
        1    0.000    0.000    0.000    0.000 {built-in method builtins.globals}
       34    0.000    0.000    0.000    0.000 {built-in method builtins.hasattr}
      292    0.000    0.000    0.000    0.000 {built-in method builtins.isinstance}
      120    0.000    0.000    0.000    0.000 {built-in method builtins.len}
        3    0.000    0.000    0.000    0.000 {built-in method builtins.max}
        4    0.000    0.000    0.000    0.000 {built-in method builtins.repr}
        1    0.000    0.000    0.000    0.000 {built-in method builtins.setattr}
        3    0.000    0.000    0.000    0.000 {built-in method from_bytes}
        1    0.000    0.000    0.000    0.000 {built-in method io.open_code}
        1    0.000    0.000    0.000    0.000 {built-in method marshal.loads}
        1    0.000    0.000    0.000    0.000 {built-in method nt._path_splitroot}
        3    0.000    0.000    0.000    0.000 {built-in method nt.fspath}
        3    0.000    0.000    0.000    0.000 {built-in method nt.stat}
        1    0.002    0.002    0.002    0.002 {built-in method pythoncom.MkParseDisplayName}
      116    0.001    0.000    0.001    0.000 {method 'Bind' of 'PyITypeComp' objects}
        1    0.000    0.000    0.000    0.000 {method 'BindToObject' of 'PyIMoniker' objects}
        6    0.000    0.000    0.000    0.000 {method 'GetDocumentation' of 'PyITypeInfo' objects}
        1    0.000    0.000    0.000    0.000 {method 'GetIDsOfNames' of 'PyIDispatch' objects}
       35    0.000    0.000    0.000    0.000 {method 'GetNames' of 'PyITypeInfo' objects}
        6    0.000    0.000    0.000    0.000 {method 'GetRefTypeInfo' of 'PyITypeInfo' objects}
       60    0.000    0.000    0.000    0.000 {method 'GetTypeAttr' of 'PyITypeInfo' objects}
       24    0.000    0.000    0.000    0.000 {method 'GetTypeComp' of 'PyITypeInfo' objects}
       24    0.004    0.000    0.004    0.000 {method 'GetTypeInfo' of 'PyIDispatch' objects}
       29    0.001    0.000    0.001    0.000 {method 'Invoke' of 'PyIDispatch' objects}
        6    0.001    0.000    0.001    0.000 {method 'InvokeTypes' of 'PyIDispatch' objects}
       20    0.223    0.011    0.223    0.011 {method 'Next' of 'PyIEnumVARIANT' objects}
        5    0.000    0.000    0.000    0.000 {method 'QueryInterface' of 'PyIUnknown' objects}
        9    0.000    0.000    0.000    0.000 {method '__contains__' of 'frozenset' objects}
        1    0.000    0.000    0.000    0.000 {method '__exit__' of '_io._IOBase' objects}
        1    0.000    0.000    0.000    0.000 {method '__exit__' of '_thread.RLock' objects}
        2    0.000    0.000    0.000    0.000 {method '__exit__' of '_thread.lock' objects}
       21    0.000    0.000    0.000    0.000 {method 'append' of 'list' objects}
        1    0.000    0.000    0.000    0.000 {method 'copy' of 'dict' objects}
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
       21    0.000    0.000    0.000    0.000 {method 'endswith' of 'str' objects}
        1    0.000    0.000    0.000    0.000 {method 'format' of 'str' objects}
      167    0.000    0.000    0.000    0.000 {method 'get' of 'dict' objects}
       20    0.000    0.000    0.000    0.000 {method 'join' of 'str' objects}
        2    0.000    0.000    0.000    0.000 {method 'keys' of 'dict' objects}
        1    0.000    0.000    0.000    0.000 {method 'lower' of 'str' objects}
        1    0.000    0.000    0.000    0.000 {method 'pop' of 'dict' objects}
        1    0.000    0.000    0.000    0.000 {method 'read' of '_io.BufferedReader' objects}
        2    0.000    0.000    0.000    0.000 {method 'replace' of 'str' objects}
        4    0.000    0.000    0.000    0.000 {method 'rfind' of 'str' objects}
       11    0.000    0.000    0.000    0.000 {method 'rpartition' of 'str' objects}
        1    0.000    0.000    0.000    0.000 {method 'rsplit' of 'str' objects}
       17    0.000    0.000    0.000    0.000 {method 'rstrip' of 'str' objects}
       45    0.000    0.000    0.000    0.000 {method 'startswith' of 'str' objects}

Please let me know what you think and if you would like a PR to be created to address this issue.

[dekuNukem]

Great job! I have to admit I'm not very familar with windows-specific side of things, and I just grabbed the first thing that worked! 😅

Feel free to submit a PR, I'll try it out!

[dekuNukem]

Maybe I'll incorporate the fix myself over the weekend as well 😅