Installing Debug build MSI leads to ebpfsvc.exe not being able to start

[lmb]

Describe the bug

After installing from a Debug build MSI, the eBPF service fails to start with error 1053.

OS information

No response

Steps taken to reproduce bug

1. Install Debug MSI via setup-ebpf.ps1
2. Open services.msc
3. Try to start the service

Expected behavior

The service should start.

Actual outcome

It doesn't.

Additional details

I think the problem lies in the fact that the service is configured to run as Local Service, which doesn't have access to the VC runtime debug dlls in system32. These are copied there by setup-ebpf.ps1. Copying the dlls into the JIT folder makes things work again.

[shankarseal]

First, change the setup script to copy the DBG CRT locally. Second, please investigate if/why the service fails to load when the CRT is in system32 directory.

[vpidatala94]

attempting local reproduction

[vpidatala94]

setup script excepts the CRT DLL in the same location, thus copy action is a pre-step to the execution of this script hence not needed.

[vpidatala94]

[vpidatala94]

[vpidatala94]

[vpidatala94]

Observation when setting - logon as local system account the issue is not seen -

[vpidatala94]

Seeing this error when using the installer to install ebpf with JIT -

[vpidatala94]

- this shows that LOCAL SERVICE does not have read access to the debug dll "ending with d.dll", the reason being LOCAL SERVICE is part of Users group which is not granted read access for d.dll such as msvcp140d.dll in C:\Windows\System32 directory

[shpalani]

You need JIT installed for Debug msi, for running Kernel programs, along with vc_redist.x64.exe

[lmb]

Could we copy the debug dlls to the JIT directory as a work around? (Side note: is the user expected to install the debug runtime when using an MSI without going via setup-ebpf.ps1?)

[lmb]

Removing the triage label because I'd like to discuss this in tomorrows meeting.

[shankarseal]

@vpidatala94 - please make changes to statically link CRT in debug bits.

[vpidatala94]

[vpidatala94]

this is after statically linking the debug runtime libraries to ebpf-for-windows.msi

[vpidatala94]

[vpidatala94]

[vpidatala94]

static linking will cause breakages in a scenario where - the customer is a using a release version of the product and debug version of the EBPF product. Fix addressed parallelly here - https://github.com/microsoft/ebpf-for-windows/pull/3957, as part of this bug updating to documentation to help customers know that if they are using debug MSI to install ebpf directly they are supposed to copy the Debug DLL to a detectable location to allow it to be loaded

[lmb]

@vpidatala94 if I understand you correctly you’ve tried running an application which is linked against the release runtime against an ebpfapi.dll statically linked against the debug runtime. Does this even work when dynamically linking the debug instead? I imagine that this might not be supported in the first place?

[lmb]

The following makes me think that mixing release and debug versions isn't really supported in the first place:

All modules passed to a given invocation of the linker must have been compiled with the same run-time library compiler option (/MD, /MT, /LD).

So the use case of a customer using the release libs with debug ebpf for windows isn't really a valid use case? That would mean we can statically link after all.

[vpidatala94]

Apologies for the delay, the problem with static linking CRT is the ebpfapi.dll will have a different context compared to the Custom application (both using different sets of CRT) causing state difference/incompatibility as a result the ebpfapi.dll CRT will not be able to find the FD that customer application attempts to close. Based on discussion with Dave, the new direction we are going forth is making the debug installer more "complete" that the installer will will attempt to copy the DEBUG CRT to a set location included in path and if these are DEBUG CRT are not found, the installer will raise a warning.

[shankarseal]

@vpidatala94 please investigate if static linking against CRT works for all scenarios ( a C application loading ebpf api.dll etc.) If things turn out OK, when you change to static CRT, please revert #4014.