BugId fails with error: Unrecognized !heap output

[maxcoderrrr]

Hi,

While trying to fix https://github.com/SkyLined/BugId/issues/121, I came across another bug that occurs when BugId is used with the latest version of the Debugging Tools for Windows and encounters a crash. The original !heap commands got replaced with !ext.heap, which triggers an assertion in BugId due to the unexpected output:

┌───[ Software license warning ]───────────────────────────────────────────────────────────────────────────────────────
│ ▲ You have no license for BugId and your trial period will expire on December 27th, 2023
│ ▲ You have no license for mBugId and your trial period will expire on December 27th, 2023
└──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
→ Running process ids: 8724
+ Main process 8724/0x2214 (foo.exe, x86, IL:2): Attached ("C:\test\foo.exe" "C:\test\foo.bar").
┌───[ Fatal builtins.AssertionError Exception in thread 1552/0x610 (cThread#1C83E15D010{main = __fRun, #1552, running}) ]─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
│ cProcess.fo0GetHeapManagerDataForAddress: Unrecognized !heap output: [b"the `!heap -p' commands in exts.dll have been replaced", b'with equivalent commands in ext.dll.', b'If your are in a KD session, use `!ext.heap -p`']
│ 
│ Local variables:
│   asbCdbHeapOutput = [<instance builtins:bytes 'b"the `!heap -p\' commands in exts.dll have been replaced"'>#1C83E18B750, <instance builtins:bytes "b'with equivalent commands in ext.dll.'">#1C83E1B81C0, <instance builtins:bytes "b'If your are in a KD session, use `!ext.heap -p`'">#1C83E1B90C0]#1C83E079840
│   oProcess = <mBugId.cProcess.cProcess.cProcess object at 0x000001C83E17A1D0>#1C83E17A1D0
│   oVitualAllocation = <VirtualAllocation(Allocated, uState=MEM_COMMIT, uType=MEM_PRIVATE, base @ 0x4F0`0000, [0x4FF`E000-0x500`0000] (0x2000 bytes, uProtection = PAGE_READWRITE)) at 0x1C83E171290>#1C83E171290
│   uAddressNearHeapBlock = 83879644
│ 
│ Stack for thread 1552/0x610 (cThread#1C83E15D010{main = __fRun, #1552, running}):
│ ─┐ __fRun @ C:\bugid3\modules\mBugId\cCdbWrapper\cCdbWrapper_cHelperThread.py:66
│  │ 65:      try:
│  │ 66:        oSelf.__fActivity(*oSelf.__axActivityArguments);
│  ├─┐ cCdbWrapper_fCdbStdInOutHelperThread @ C:\bugid3\modules\mBugId\cCdbWrapper\cCdbWrapper_fCdbStdInOutHelperThread.py:54
│  ╷ │ 53:    oCdbWrapper.fbFireCallbacks("Log message", "Main loop #%d" % uMainLoopCounter);
│  ╷ │ 54:    (bEventIsFatal, bEventHasBeenHandled) = oCdbWrapper.ftbHandleLastCdbEvent(asbOutputWhileRunningApplication);
│  ╷ ├─┐ cCdbWrapper_ftbHandleLastCdbEvent @ C:\bugid3\modules\mBugId\cCdbWrapper\cCdbWrapper_ftbHandleLastCdbEvent.py:229
│  ╷ ╷ │ 228:  ### Report bug and see if the collateral bug handler can ignore it #################################################
│  ╷ ╷ │ 229:  o0BugReport.fReport();
│  ╷ ╷ ├─┐ cBugReport?.fReport @ C:\bugid3\modules\mBugId\cBugReport\cBugReport.py:261
│  ╷ ╷ ╷ │ 260:      if oSelf.bRegistersRelevant:
│  ╷ ╷ ╷ │ 261:        s0RegistersBlockHTML = oSelf.fs0GetRegistersBlockHTML(oSelf.__oProcess, oSelf.__oWindowsAPIThread);
│  ╷ ╷ ╷ ├─┐ cBugReport_fs0GetRegistersBlockHTML @ C:\bugid3\modules\mBugId\cBugReport\cBugReport_fs0GetRegistersBlockHTML.py:7
│  ╷ ╷ ╷ ╷ │ 6:  # Create and add registers block
│  ╷ ╷ ╷ ╷ │ 7:  a0txRegisters = oProcess.fa0txGetRegistersForThreadId(oWindowsAPIThread.uId);
│  ╷ ╷ ╷ ╷ ├─┐ cProcess_fa0txGetRegistersForThreadId @ C:\bugid3\modules\mBugId\cProcess\cProcess_fa0txGetRegistersForThreadId.py:19
│  ╷ ╷ ╷ ╷ ╷ │ 18:      else:
│  ╷ ╷ ╷ ╷ ╷ │ 19:        o0HeapManagerData = oProcess.fo0GetHeapManagerDataForAddressNearHeapBlock(
│  ╷ ╷ ╷ ╷ ╷ ├─┐ cProcess?.fo0GetHeapManagerDataForAddressNearHeapBlock @ C:\bugid3\modules\mBugId\cProcess\cProcess.py:234
│  ╷ ╷ ╷ ╷ ╷ ╷ │ 233:    else:
│  ╷ ╷ ╷ ╷ ╷ ╷ │ 234:      return oSelf.fo0GetWindowsHeapManagerDataForAddressNearHeapBlock(uAddressNearHeapBlock);
│  ╷ ╷ ╷ ╷ ╷ ╷ ├─┐ cProcess?.fo0GetWindowsHeapManagerDataForAddressNearHeapBlock @ C:\bugid3\modules\mBugId\cProcess\cProcess.py:245
│  ╷ ╷ ╷ ╷ ╷ ╷ ╷ │ 244:      oSelf.__do0HeapManagerData_by_uAddressNearHeapBlock[uAddressNearHeapBlock] = \
│  ╷ ╷ ╷ ╷ ╷ ╷ ╷ │ 245:          cProcess_fo0GetWindowsHeapManagerDataForAddressNearHeapBlock(oSelf, uAddressNearHeapBlock);
│  ╷ ╷ ╷ ╷ ╷ ╷ ╷ ├─┐ cProcess_fo0GetWindowsHeapManagerDataForAddressNearHeapBlock @ C:\bugid3\modules\mBugId\cProcess\cProcess_fo0GetWindowsHeapManagerDataForAddressNearHeapBlock.py:151
│  ╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ │ 150:  
│  ╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ │ 151:  assert len(asbCdbHeapOutput) >= 4, \
│  ╒═══════════════╛ ▲ Assertion failed: 'cProcess.fo0GetHeapManagerDataForAddress: Unrecognized !heap output: [b"the `!heap -p\' commands in exts.dll have been replaced", b\'with equivalent commands in ext.dll.\', b\'If your are in a KD session, use `!ext.heap -p`\']'
│  │ __fRun @ C:\bugid3\modules\mBugId\cCdbWrapper\cCdbWrapper_cHelperThread.py:74
│  │ 73:        cException, oException, oTraceBack = sys.exc_info();
│  │ 74:        if not oSelf.__oCdbWrapper.fbFireCallbacks("Internal exception", oSelf.__oThread, oException, oTraceBack):
│ ═╛ ▲ Application terminated because exception was not handled.
└─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

Please report the above details at the below web-page so it can be addressed:
    https://github.com/SkyLined/BugId/issues/new
If you do not have a github account, or you want to report this issue
privately, you can also send an email to:
    BugId@skylined.nl

In your report, please copy ALL the information about the exception reported
above, as well as the stack trace and BugId version information. This makes
it easier to determine the cause of this issue and makes for faster fixes.

If you can reproduce the issue, it would help a lot if you can run BugId in
verbose mode by adding the --verbose command-line argument.
as in: BugId -v --pause --isa=x86 --pid=8724 --handle-jit-event=572 --reports=C:\Users\admin\BugId reports

  ____________________________________________________________________________
                              __
   ││▌║█▐▐║▌▌█│║║│      _,siSP**YSis,_       ╒╦╦══╦╗             ╒╦╦╕    ╔╦╕
   ││▌║█▐▐║▌▌█│║║│    ,SP*'`    . `'*YS,      ║╠══╬╣ ╔╗ ╔╗ ╔╦═╦╗  ║║  ╔╦═╬╣
   ╵2808197631337╵   dS'  _    |    _ 'Sb    ╘╩╩══╩╝ ╚╩═╩╝ ╚╩═╬╣ ╘╩╩╛ ╚╩═╩╝
                    dP     \,-` `-<` `  Y;                 ╚╩═╩╝    ╮╷╭
      ╮╷╭          ,S`  \+' \      \    `Sissssssssssssssssssss,   :O()    ╲ö╱
     :O()          (S   (   | --====)   :SSSSSSSSSSSSSSSSSSSSSSD    ╯╵╰    ─O─
      ╯╵╰  ╮╷╭     'S,  /+, /      /    ,S?********************'           ╱O╲
           ()O:     Yb    _/'-_ _-<._.  dP
           ╯╵╰       YS,       |      ,SP         https://bugid.skylined.nl
  ____________________`Sbs,_    ' _,sdS`______________________________________
                        `'*YSissiSY*'`
                              ``
┌───[ Version information ]───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
│ ▲ BugId version: 2023-06-01 14:23 (in trial period).
│ ▲ mBugId version: 2022-12-12 12:05 (in trial period).
│ √ mConsole version: 2022-12-12 12:05.
│ √ mDateTime version: 2022-12-12 12:04.
│ √ mDebugOutput version: 2022-12-12 12:05.
│ √ mFileSystemItem version: 2022-12-12 12:05.
│ √ mHumanReadable version: 2022-12-12 12:04.
│ √ mMultiThreading version: 2022-12-12 12:05.
│ √ mNotProvided version: 2022-12-12 12:04.
│ √ mProductDetails version: 2022-12-12 12:05.
│ √ mRegistry version: 2022-12-12 12:05.
│ √ mWindowsAPI version: 2022-12-12 12:05.
│ √ mWindowsSDK version: 2022-12-12 12:04.
│ • Platform OS: Windows-10-10.0.19045-SP0 on AMD64 processor.
│ • Python version: 3.11.1, 64 bit.
│ • cdb.exe (x86) version: 10.0.22621.2428.
│ • cdb.exe (x64) version: 10.0.22621.2428.
└─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Thank you in advance for helping to improve BugId!
√ A copy of the error report can be found in C:\bugid3\Internal error reports\2023-11-27 15։33։24.109061 BugId error report #30.txt.

[gnbon]

https://github.com/SkyLined/mBugId/blob/4b10c35df1de8e86ddff4d290b65ac4829cbb59f/cProcess/cProcess_fo0GetWindowsHeapManagerDataForAddressNearHeapBlock.py#L109

This is a problem caused by the !heap command being dropped. I solved the problem by changing the !heap command to !ext.heap

[maxcoderrrr]

@gnbon Thank you! Yes, I should have mentioned the same, as it is shown in the error message too. Perhaps adding a check inside BugId on the installed WinDbg version would be good, and to execute the correct heap command according to that. Sometimes !ext.heap will not work and return blank if one is using a WinDbg version that is too new for the given system.

[SkyLined]

Thank you for the bug report and for the solution. I've implemented a fix that attempts to use "!heap" first and if it detects the error message, switches to using "!ext.heap". https://github.com/SkyLined/mBugId/commit/5944bd7f4797c4b943285fd1ed3173f22adb7872