Question: How to register RemedyBG as a JIT debugger?

[sonictk]

I've been trying for the longest time to get RemedyBG registered as a JIT debugger, so that when an application starts/crashes, that RemedyBG is automatically shown as one of the options in vsjitdebugger.exe, or forcing it to be used (like how you can do so for WinDbg)

I've tried following the steps from the official MSDN documentation regarding this, but I haven't had any success getting it to work.

https://learn.microsoft.com/en-us/visualstudio/debugger/debug-using-the-just-in-time-debugger?view=vs-2022

http://etutorials.org/Programming/programming+microsoft+visual+c+sharp+2005/Part+IV+Debugging/Chapter+13+Advanced+Debugging/Just-In-Time+JIT+Debugging/

Has anyone gotten this to work successfully for RemedyBG? It's very important for being able to attach to executables that are launched automatically as part of another process and halting them immediately on startup; being able to customize JIT debugging for this would be a huge deal for being able to use RemedyBG to troubleshoot those scenarios.

[x13pixels]

I'm able to get it to work with the following registry keys:

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\AeDebug
  "Debugger" (String) = c:\full\path\to\remedybg.exe attach-to-process-by-id %ld
  "Auto" (DWORD) = 1

The experience isn't ideal (e.g., launches RemedyBG twice in the test app I was using). I'll leave this issue open to get these sorts of things addressed.

[sonictk]

Hmm, if I try to use the same command-line options for

Computer\HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Image File Execution Options\myprocess.exe

And then launch myprocess.exe, it doesn't seem to work; it just crashes RemedyBG on start. I tried just putting the executable path to RemedyBG without any other command-line arguments and that also didn't work, though at least it just launched RemedyBG (but didn't attach to the process nor do anything else).

The default value is "C:\Windows\system32\vsjitdebugger.exe", which allows you to choose from different Visual Studio versions to debug the executable with. I don't know if it's somehow possible to allow RemedyBG to be registered as a JIT debugger to show up in that list to be chosen from, but that would be really cool. It doesn't like like MSFT makes this easy, though, as this was all that I could find on the topic: https://groups.google.com/g/microsoft.public.win32.programmer.kernel/c/R17MIcMMtjU#818bdd912773d457

[charlesastaylor]

I'm able to get it to work with the following registry keys:

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\AeDebug
  "Debugger" (String) = c:\full\path\to\remedybg.exe attach-to-process-by-id %ld
  "Auto" (DWORD) = 1

The experience isn't ideal (e.g., launches RemedyBG twice in the test app I was using). I'll leave this issue open to get these sorts of things addressed.

Like I said on discord, setting the keys like you have doesn't work for me. But having found this doc page I was curious whether the double launching was down to the event handle it mentions. By just directly replacing the vsjitdebugger.exe with my own program I was able to reproduce the double opening, and then by signalling the event in my program the double open went away.

eg:

s32 main(s32 num_args, char* args[]) {
    // Default command line for jit: "C:\WINDOWS\system32\vsjitdebugger.exe" -p %ld -e %ld
    if (num_args == 5) {
        // char *pid = args[2];
        char *event_handle_string = args[4];       
        s64 x = atoll(event_handle_string);
        HANDLE event_handle = (HANDLE)x;
        BOOL result = SetEvent(event_handle);
        if (result) {
            printf("Success!\n");
        } else {
            printf("Failed!\n");
        }
    } else {
        printf("Yo!\n");
    }
    getchar();
}

[pmttavara]

by signalling the event in my program the double open went away.

What does this mean, practically, for RemedyBG? Are you saying that Remedy should, for example, accept a flag named -jit-debug-event <integer> and then call SetEvent on that integer?

[charlesastaylor]

by signalling the event in my program the double open went away.

What does this mean, practically, for RemedyBG? Are you saying that Remedy should, for example, accept a flag named -jit-debug-event <integer> and then call SetEvent on that integer?

I believe so yep.

[x13pixels]

[0.3.8.6] Added --jit-event-handle option to attach-process-by-id and signal the given event after successfully attaching to the process.

Leaving this open, however, for tying into the vsjitdebugger dialog box.