Default service name conflicts with Microsoft (pmem)

[wallrik]

We have spent some time trying to understand how WinPmem interacts with Microsofts persistent memory driver. (Some random docs mentioning it: HCI, SQL, PS)

Unfortunately, WinPmem completely ignores it, removes the service, and does not restore it.

Also, if the driver is already loaded, the service cannot be stopped, and WinPmem fails to run.

This should only be an issue on Windows 10 or newer.

C:\>sc qc pmem
[SC] QueryServiceConfig SUCCESS

SERVICE_NAME: pmem
        TYPE               : 1  KERNEL_DRIVER
        START_TYPE         : 3   DEMAND_START
        ERROR_CONTROL      : 1   NORMAL
        BINARY_PATH_NAME   : \SystemRoot\System32\drivers\pmem.sys
        LOAD_ORDER_GROUP   :
        TAG                : 0
        DISPLAY_NAME       : Microsoft persistent memory disk driver
        DEPENDENCIES       :
        SERVICE_START_NAME :

I would suggest, instead of making WinPmem test different cases if the driver is loaded or not, and trying to restore it, I would just rename the WinPmem service to winpmem instead of pmem.

Also, a command-line argument for the service name would also be nice, as an optional extra. But not a high priority.

Also, WinPmem as a name, is that for physical memory? Because imo, that's assumed, in my mind. Pmem means persistent memory. But that's not what I'm creating this issue for! 😁

[vivianezw]

I wasn't aware of that. This is the first time report. But it seems unavoidable to get a new name, then. "winpmem" instead of "pmem" would be an obvious choice. It is likely to create backward compability problems with existing tools/frameworks. I guess we have to discuss this. The fix itself would be a really tiny action that doesn't cost more than 5 minutes.

The reason for the name "pmem" is the abbreviation for "physical memory" and has been used by Winpmem for quite a while.

[scudette]

We used to actually use a random service name to avoid anti forensics in the past. Maybe we should consider reintroducing that?

[scudette]

just to clarify is this a clash of service name or device name? The device name is hard coded in the driver so we cant change it without rebuilding and resigning the driver.

[wallrik]

just to clarify is this a clash of service name or device name? The device name is hard coded in the driver so we cant change it without rebuilding and resigning the driver.

The reason I stumbled upon it was because WinPmem failed to run on a server without any obvious error. I think it's just an issue with the service name. Somehow, it had failed to install, and Microsofts pmem had been started instead. This is on a server that did not have any persistent memory, so the service/driver shouldn't even have been running.

But the issue then was, once its started, it cannot be stopped. This prevents WinPmem from functioning, and it doesn't really give any errors. It just says it extracts the driver, tries to do something, den deletes the driver. Nothing else happens.

C:\Windows\Temp>winpmem_mini_x64_rc2.exe dump.raw
WinPmem64
Extracting driver to C:\Users\user\AppData\Local\Temp\pmeE3C5.tmp
Driver Unloaded.
Deleting C:\Users\user\AppData\Local\Temp\pmeE3C5.tmp
Driver Unloaded.

C:\Windows\Temp>sc query pmem

SERVICE_NAME: pmem
        TYPE               : 1  KERNEL_DRIVER
        STATE              : 4  RUNNING
                                (NOT_STOPPABLE, NOT_PAUSABLE, IGNORES_SHUTDOWN)
        WIN32_EXIT_CODE    : 0  (0x0)
        SERVICE_EXIT_CODE  : 0  (0x0)
        CHECKPOINT         : 0x0
        WAIT_HINT          : 0x0

C:\Windows\Temp>sc qc pmem
[SC] QueryServiceConfig SUCCESS

SERVICE_NAME: pmem
        TYPE               : 1  KERNEL_DRIVER
        START_TYPE         : 4   DISABLED
        ERROR_CONTROL      : 1   NORMAL
        BINARY_PATH_NAME   : System32\drivers\pmem.sys
        LOAD_ORDER_GROUP   :
        TAG                : 0
        DISPLAY_NAME       : Microsoft persistent memory disk driver
        DEPENDENCIES       :
        SERVICE_START_NAME :

[wallrik]

We used to actually use a random service name to avoid anti forensics in the past. Maybe we should consider reintroducing that?

There are definitely pros and cons, for sure. But I think that I would prefer a well-known name.

Looking at my currently installed services/drivers there seem to be some patterns. The ones that don't just use a simple name tend to use a suffix like Svc, Dev or Drv.

Then there's the per-user services that use <service name>_LUID such as CDPUserSvc_4f503 and BluetoothUserService_4f503. But those are not drivers.

PsExec always installs its service as PSEXESVC. But that's not for a driver, that's just to gain SYSTEM.

Sysmon's monitor driver installs itself as SysmonDrv.

I guess PmemDrv would work. But it's very close still to Microsofts pmem.

just to clarify is this a clash of service name or device name? The device name is hard coded in the driver so we cant change it without rebuilding and resigning the driver.

To answer that again, I should clarify that I don't really know 😐

[vivianezw]

For the servicename, Winpmem generates a random name using the pattern "pme####", where #### is some random term. So, if the filename is "pmeE3C5.tmp", then the servicename is pmeE3C5. pmeE3C5.tmp is the driverfile, and pmeE3C5 the servicename, in this case.

Since winpmem unloaded immediately, something must have gone wrong (but unrelated, I think). You will not find it using sc, because it already unloaded and deleted itself. You can use sc only while it didn't unload. The service name would have bee pmeE3C5 in your example above, e.g., "sc query pmeE3C5", but only if not unloaded. (correction: sc uses the display name, not the kernel module name, and our display name is actually really "pmem", and this isn't good).

@edit: the device name is hardcoded as "pmem", the service name is randomized, and the file name is randomized, but the display name is "pmem". Latter is the probable cause for trouble!

[vivianezw]

Ist just tried myself. I can see why you think it's clashing with MS pmem.sys, and I now tend to think the same. sc uses the display name!

I did the sc command (I normally don't do this). It loads fine, and the path points to the randomized filepath string. Also, everything else looks fine, but what I see is that the display name (in sc) is pmem. This may create a conflict. On my system, it doesn't, but on yours, it probably does. sc uses the display name, and certain other things on the OS may also use it.

[vivianezw]

Okay, I did a name conflict resolve branch (see second branch), changed service / display name to "winpmem" (from "pmem"), and tested it. Now in sc, I don't get anything anymore for sc query pmem (which was the case before), and it's now sc query winpmem that works. I think this should hopefully resolve the name conflict.

Only the usermode program changed (and one string, with one line). I could append my minitool version here and you can try if this helped, or you compile it yourself.

[wallrik]

Only the usermode program changed (and one string, with one line). I could append my minitool version here and you can try if this helped, or you compile it yourself.

I would love to try it out, if you could attach a compiled version! 👍 (saves me a lot of trouble)

Now in sc, I don't get anything anymore for sc query pmem (which was the case before) [...]

The service should present on any recent Windows 10+ system. I don't know in what release it was introduced. Maybe you've accidentally removed yours before?

Regarding sc using the service name or display name, I'm pretty sure it's the service name. sc query W32Time works, but sc query "Windows Time" with or without quotes does not work.

PS > Get-WinEvent -LogName System -FilterXPath "*[System[(EventID=7045)]]" -MaxEvents 1 | Select -ExpandProperty Message
A service was installed in the system.

Service Name:  pmem
Service File Name:  C:\Users\user\AppData\Local\Temp\pmeB689.tmp
Service Type:  kernel mode driver
Service Start Type:  demand start
Service Account:

[wallrik]

I saw the change in the other branch, and that made me curious to read some more code. (I haven't really done a deep dive before) This part here caught my attention, in relation to service name vs display name:

https://github.com/Velocidex/WinPmem/blob/59889475fd6341961caf9f6b75f7348b691109f8/src/executable/winpmem.cpp#L661-L673

That can probably be changed to set the display name to NULL, as it doesn't really do anything except make it a little bit confusing which value is which, since they are set to the same thing. lpDisplayName is optional.

Either way, just a minor thing. 😊 Doesn't really make a difference for me as a user.

[vivianezw]

So it was the servicename, in the end. Hopefully this is the issue. Here is the minitool, using servicename winpmem now. I would like to know if this works?

winpmem_minitool_6June2024_test.zip

[wallrik]

Here is the minitool, using servicename winpmem now. I would like to know if this works?

That's perfect! It works very well. Even when pmem.sys is already loaded, there are no issues.

Everything is working as expected. Very good! 🥳

C:\Users\user\Desktop>sc qc pmem
[SC] QueryServiceConfig SUCCESS

SERVICE_NAME: pmem
        TYPE               : 1  KERNEL_DRIVER
        START_TYPE         : 3   DEMAND_START
        ERROR_CONTROL      : 1   NORMAL
        BINARY_PATH_NAME   : \SystemRoot\System32\drivers\pmem.sys
        LOAD_ORDER_GROUP   :
        TAG                : 0
        DISPLAY_NAME       : Microsoft persistent memory disk driver
        DEPENDENCIES       :
        SERVICE_START_NAME :

C:\Users\user\Desktop>sc query pmem

SERVICE_NAME: pmem
        TYPE               : 1  KERNEL_DRIVER
        STATE              : 4  RUNNING
                                (NOT_STOPPABLE, NOT_PAUSABLE, IGNORES_SHUTDOWN)
        WIN32_EXIT_CODE    : 0  (0x0)
        SERVICE_EXIT_CODE  : 0  (0x0)
        CHECKPOINT         : 0x0
        WAIT_HINT          : 0x0

C:\Users\user\Desktop>winpmem_minitool_6June2024_test.exe mem.dmp
WinPmem64
Extracting driver to C:\Users\user\AppData\Local\Temp\win2FF5.tmp
Driver Unloaded.
Loaded Driver C:\Users\user\AppData\Local\Temp\win2FF5.tmp.
Deleting C:\Users\user\AppData\Local\Temp\win2FF5.tmp
The system time is: 12:08:07
Will generate a RAW image
 - buffer_size_: 0x1000
CR3: 0x00001AD000
 5 memory ranges:
Start 0x00002000 - Length 0x0009E000
Start 0x00100000 - Length 0x0E901000
Start 0x0EA02000 - Length 0x0116D000
Start 0x0FBFF000 - Length 0xB0401000
Start 0x100000000 - Length 0x340000000
max_physical_memory_ 0x440000000
Acquitision mode PTE Remapping
Padding from 0x00000000 to 0x00002000
pad
 - length: 0x2000

00% 0x00000000 .
copy_memory
 - start: 0x2000
 - end: 0xa0000

00% 0x00002000 .
Padding from 0x000A0000 to 0x00100000
pad
 - length: 0x60000

00% 0x000A0000 .
copy_memory
 - start: 0x100000
 - end: 0xea01000

00% 0x00100000 ...............
Padding from 0x0EA01000 to 0x0EA02000
pad
 - length: 0x1000

01% 0x0EA01000 .
copy_memory
 - start: 0xea02000
 - end: 0xfb6f000

01% 0x0EA02000 ..
Padding from 0x0FB6F000 to 0x0FBFF000
pad
 - length: 0x90000

01% 0x0FB6F000 .
copy_memory
 - start: 0xfbff000
 - end: 0xc0000000

01% 0x0FBFF000 ..................................................
06% 0x41BFF000 ..................................................
10% 0x73BFF000 ..................................................
15% 0xA5BFF000 ...........................
Padding from 0xC0000000 to 0x100000000
pad
 - length: 0x40000000

17% 0xC0000000 ..................................................
17% 0xC0000000 ..............
copy_memory
 - start: 0x100000000
 - end: 0x440000000

23% 0x100000000 ..................................................
28% 0x132000000 ..................................................
32% 0x164000000 ..................................................
37% 0x196000000 ..................................................
41% 0x1C8000000 ..................................................
46% 0x1FA000000 ..................................................
51% 0x22C000000 ..................................................
55% 0x25E000000 ..................................................
60% 0x290000000 ..................................................
64% 0x2C2000000 ..................................................
69% 0x2F4000000 ..................................................
74% 0x326000000 ..................................................
78% 0x358000000 ..................................................
83% 0x38A000000 ..................................................
87% 0x3BC000000 ..................................................
92% 0x3EE000000 ..................................................
97% 0x420000000 ................................
The system time is: 12:08:27
Driver Unloaded.

C:\Users\user\Desktop>sc qc pmem
[SC] QueryServiceConfig SUCCESS

SERVICE_NAME: pmem
        TYPE               : 1  KERNEL_DRIVER
        START_TYPE         : 3   DEMAND_START
        ERROR_CONTROL      : 1   NORMAL
        BINARY_PATH_NAME   : \SystemRoot\System32\drivers\pmem.sys
        LOAD_ORDER_GROUP   :
        TAG                : 0
        DISPLAY_NAME       : Microsoft persistent memory disk driver
        DEPENDENCIES       :
        SERVICE_START_NAME :

PS > Get-WinEvent -LogName System -FilterXPath "*[System[(EventID=7045)]]" -MaxEvents 1 | Select -ExpandProperty Message
A service was installed in the system.

Service Name:  winpmem
Service File Name:  C:\Users\user\AppData\Local\Temp\win2FF5.tmp
Service Type:  kernel mode driver
Service Start Type:  demand start
Service Account:

[vivianezw]

Good to hear! The thing I don't like is how the randomized pattern "win####" turns out now.

It fixes the name conflict, for now that's what matters. But the minitool could use some overhaul, some time in future.

[wallrik]

Good to hear! The thing I don't like is how the randomized pattern "win####" turns out now.

It fixes the name conflict, for now that's what matters. But the minitool could use some overhaul, some time in future.

You don't like the random name? 😅 Haha. Well, yeah, a random name starting with win can mean pretty much anything, I guess.

https://github.com/Velocidex/WinPmem/blob/4d27adddb854e9151233731f555fb11e42f80bbb/src/executable/winpmem.cpp#L849-L850

I guess it would be easy to change it (lpPrefixString). Personally, I wouldn't mind if it's not random at all, and is just named winpmem_x64.sys. But it would be easier for malicious actors to detect when it's loaded.

[vivianezw]

Now Scudette needs to make a new release. ^^

I don't know, there's pro and contra, I leave this up to Scudette. For now it's not important enough. ;-D

[scudette]

I tried to rebuild the C imager and it seems very outdated and difficult to maintain - So I wrote a new Go based imager on the releases page. It is much simpler for us to maintain going forward and also supports some compression out of the box (as well as allowing the service name to be changed). Please test and see if it solves this issue.

[wallrik]

Very exciting stuff! Made me want to do work related stuff on a Saturday 😁

It works perfectly fine! Thank you very much. 🙏 (syntax change tripped me up though)

C:\Users\user\Desktop>sc query pmem

SERVICE_NAME: pmem
        TYPE               : 1  KERNEL_DRIVER
        STATE              : 4  RUNNING
                                (NOT_STOPPABLE, NOT_PAUSABLE, IGNORES_SHUTDOWN)
        WIN32_EXIT_CODE    : 0  (0x0)
        SERVICE_EXIT_CODE  : 0  (0x0)
        CHECKPOINT         : 0x0
        WAIT_HINT          : 0x0

C:\Users\user\Desktop>go-winpmem_amd64_1.0-rc1.exe memory.dmp
go-winpmem_amd64_1.0-rc1.exe: error: expected command but got "memory.dmp", try --help

C:\Users\user\Desktop>go-winpmem_amd64_1.0-rc1.exe acquire memory.dmp
Writing driver to C:\Users\user\AppData\Local\Temp\1695657030.sys
Creating service winpmem
Installed service winpmem
Started service winpmem
Memory Info:

CR3: 0x1ad000
NtBuildNumber: 0x4a64
KernelBase: 0xfffff80439000000
KPCR:
- 0xfffff80437262000
- 0xffffc78051c67000
- 0xffffc78051d2a000
- 0xffffc78051dad000
- 0xffffc78051ea5000
- 0xffffc78051f28000
NtBuildNumberAddr: 0xfffff80439c12030
Run:
- BaseAddress: 0x2000
  NumberOfBytes: 0x9e000
- BaseAddress: 0x100000
  NumberOfBytes: 0xe901000
- BaseAddress: 0xea02000
  NumberOfBytes: 0x116d000
- BaseAddress: 0xfbff000
  NumberOfBytes: 0xb0401000
- BaseAddress: 0x100000000
  NumberOfBytes: 0x340000000

Setting sparse output file memory.dmp
Padding 2 pages from 0x0
Copying 158 pages (0x9e000) from 0x2000
Padding 96 pages from 0xa0000
Copying 59649 pages (0xe901000) from 0x100000
Padding 1 pages from 0xea01000
Copying 4461 pages (0x116d000) from 0xea02000
Padding 144 pages from 0xfb6f000
Copying 721921 pages (0xb0401000) from 0xfbff000
Padding 262144 pages from 0xc0000000
Copying 3407872 pages (0x340000000) from 0x100000000
Completed imaging in 25.3459441s
Stopped service winpmem
Removing driver from C:\Users\user\AppData\Local\Temp\1695657030.sys

C:\Users\user\Desktop>sc qc pmem
[SC] QueryServiceConfig SUCCESS

SERVICE_NAME: pmem
        TYPE               : 1  KERNEL_DRIVER
        START_TYPE         : 3   DEMAND_START
        ERROR_CONTROL      : 1   NORMAL
        BINARY_PATH_NAME   : \SystemRoot\System32\drivers\pmem.sys
        LOAD_ORDER_GROUP   :
        TAG                : 0
        DISPLAY_NAME       : Microsoft persistent memory disk driver
        DEPENDENCIES       :
        SERVICE_START_NAME :

C:\Users\user\Desktop>sc query pmem

SERVICE_NAME: pmem
        TYPE               : 1  KERNEL_DRIVER
        STATE              : 4  RUNNING
                                (NOT_STOPPABLE, NOT_PAUSABLE, IGNORES_SHUTDOWN)
        WIN32_EXIT_CODE    : 0  (0x0)
        SERVICE_EXIT_CODE  : 0  (0x0)
        CHECKPOINT         : 0x0
        WAIT_HINT          : 0x0

Get-WinEvent -LogName System -FilterXPath "*[System[(EventID=7045)]]" -MaxEvents 1 | Select -ExpandProperty Message
A service was installed in the system.

Service Name:  winpmem
Service File Name:  C:\Users\user\AppData\Local\Temp\1695657030.sys
Service Type:  kernel mode driver
Service Start Type:  demand start
Service Account:

🥳 Woop woop.

If you want to idiot-proof it for the future, maybe it could check for naming conflicts? I did try the new --service_name parameter intentionally messing up the system. But hey, the brute-force way is good enough for now! It fixes my initial issue with the default name! 🤗

C:\Windows\system32>sc query pmem

SERVICE_NAME: pmem
        TYPE               : 1  KERNEL_DRIVER
        STATE              : 4  RUNNING
                                (NOT_STOPPABLE, NOT_PAUSABLE, IGNORES_SHUTDOWN)
        WIN32_EXIT_CODE    : 0  (0x0)
        SERVICE_EXIT_CODE  : 0  (0x0)
        CHECKPOINT         : 0x0
        WAIT_HINT          : 0x0

C:\Windows\system32>go-winpmem_amd64_1.0-rc1.exe acquire --service_name="pmem" test.dmp
Writing driver to C:\Users\user\AppData\Local\Temp\2962131492.sys
Could not stop service pmem: could not send control=1: The requested control is not valid for this service.
Creating service pmem
Could not stop service pmem: could not send control=1: The requested control is not valid for this service.
Removing driver from C:\Users\user\AppData\Local\Temp\2962131492.sys
go-winpmem_amd64_1.0-rc1.exe: error: acquire: The system cannot find the file specified.

Reboot machine after that... 💀

C:\Windows\system32>sc query pmem
[SC] EnumQueryServicesStatus:OpenService FAILED 1060:

The specified service does not exist as an installed service.

[scudette]

Thanks for testing - as you pointed out the syntax is a bit different as there is also an extract command to be able to decompress the compressed image later. I found empirically that S2 compression is slightly faster than raw image (due to less IO) and takes significantly less space.

As you can see with no compression we write a sparse file on NTFS which helps keep disk use down a bit already.