`probe` fails100% of pages while `pagedisplay` works

[tandasat]

I am trying to use PCILeech with PCIeScreamer R2 through Thunderbolt (USB-C) but seeing consistent 100% read failure as below.

>pcileech.exe probe -device fpga

 Memory Map:
 START              END               #PAGES

 Current Action: Probing Memory
 Access Mode:    Normal
 Progress:       16384 / 16384 (100%)
 Speed:          184 MB/s
 Address:        0x0000000400001000
 Pages read:     0 / 4194305 (0%)
 Pages failed:   4194305 (100%)
Memory Probe: Completed.

If I read a single page, it does work.

>pcileech.exe pagedisplay -min 0x0000

Memory Display: Contents for address: 0x0000000000000000
0000    32 7c e2 53 00 00 00 00  00 00 00 00 00 00 00 00   2|.S............
0010    0b 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00   ................
0020    00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00   ................

NB: This output is confirmed to be correct with other tools like busybox

I have been trying probe against the target running Ubuntu 18.04 and Windows 10 19H1 on the same laptop but it always fails to read 100% of pages. What could be the cause and solution?

I also notice the target crashes (panics or bug checks immediately after probe ends). Any advice would be very appreciated.

---

The below is the basic configuration information:

I use the pre-compiled pcileech executable file v4.4-20200316 and pre-compiled PCIeScreamer firmware v4.2.

The host connects to PCIeScreamer via the USB-C-to-A adapter:

The target connects to PCIeScreamer via the PCIe-to-USB-C adapter:

On UEFI settings, thunderbolt is set to "No Security". Against Ubuntu and Windows, I always try after login, and Virtualization Base Security and Kernel DMA Protection on Windows are both disabled. On the target Windows, the device is recognized to Thunderbolt port 1.

I have not been able to test against / with other device and PCIe slots as I do not have other devices I can try.

My suspicion is about a stability issue with a large volume of data transferring as I can read a single page.

[ufrisk]

it's good to see that it's working when doing a small read; that means it's working in theory and that it's probably just that my default timings are too aggressive.

You may specify new timings, and also adjust the PCIe gen down to 1 if you should so wish in the device string as per: https://github.com/ufrisk/LeechCore/wiki/Device_FPGA

example: pcileech.exe probe -device fpga://pciegen=1,tmread=1000,tmwrite=1000,tmprobe=1000,readsize=0x10000 which will give you bad performance though.

you may adjust the values, or remove options to revert back to defaults, to achieve your goals.

---

the crash with the probe command is probably that your computer have slightly crappy firmware or a device what really dislikes someone reading from its memory mapped memory. I'm afraid that it's most likely to crash the computer whenever there is a read from that specific problematic physical address. In the next version of PCILeech I will add the possibility to set a user-defined physical memory map to avoid this. MemProcFS is already most likely avoiding problematic memory.

Also, the PCIeScreamer R02 is quite alright stability wise, but for the best stability the ScreamerM2 board is recommended. This is not your issue though.

[tandasat]

Hi, Thank you for the quick reply. Unfortunately, using gen1 and adding delay does not appear to work. I eventually tested a huge delay and a very small read size but the result is the same.

>pcileech.exe probe -device fpga://pciegen=1,tmread=1000000,tmwrite=1000000,tmprobe=1000000,readsize=1024,algo=2

 Memory Map:
 START              END               #PAGES

 Current Action: Probing Memory
 Access Mode:    Normal
 Progress:       2160 / 16384 (13%)
 Speed:          3 MB/s
 Address:        0x0000000087000000
 Pages read:     0 / 4194305 (0%)
 Pages failed:   552960 (13%)
CTRL+C detected - shutting down ...

This might not be only an issue of interval, and perhaps, some differences between probe and pagedisplay (for example, internal state got messed up and any read fails with probe). I am just guessing here.

For my need, I just need to experiment few things with DMA and I can perhaps work with pagedisplay. I mostly thought there should have been some workaround.

[ufrisk]

Hrm, mbe the probe stops working if it reads some unreadable memory; I don't exactly know. Also the only value of the timings that directy affects the probe command is the tmprobe value.

Can you try the dump command instead, without running probe first?

example: pcileech.exe dump -min 0x1000 -v -vv fpga://tmread=1500,tmwrite=1500,readsize=0x4000

and try to go from there if it works somewhat?

[tandasat]

Hi, it looks like you are right about unreadable memory, but that is not limited to probe. It fails to read any address after encountering the read error until I reboot the target.

I found that if I read single page at once, and up to 0xa0000, it succeeds reading all pages.

>pcileech.exe dump -min 0x0000 -max 0xa0000 -v -vv -device fpga://tmread=1500,tmwrite=1500,readsize=0x1000

However, if we try to read more, it fails, and then any dump and displaypage fails after that.

>pcileech.exe dump -min 0x0000 -max 0xa1000 -v -vv -device fpga://tmread=1500,tmwrite=1500,readsize=0x1000
...
 Current Action: Dumping memory
 Access Mode:    Normal
 Progress:       0 / 0 (100%)
 Speed:          644 kB/s
 Address:        0x00000000000A1000
 Pages read:     160 / 161 (99%)
 Pages failed:   1 (0%)

This is reproduceable with pagedisplay

>pcileech.exe pagedisplay -min 0x9f000   # this works
>pcileech.exe pagedisplay -min 0xa0000   # attempt to touch unreadable memory
Memory Display: Failed reading memory at address: 0x00000000000A0000.
>pcileech.exe pagedisplay -min 0x9f000   # no longer works
Memory Display: Failed reading memory at address: 0x000000000009F000.

It is probably specific to my target or the way I connected devices. I'd think unreadable physical memory addresses are everywhere so this problem should have reported a lot more if it is generic. Nonetheless, I am happy to look into with you the cause if you are interested in and have ideas to try out on my side. For now, I am ok to close this as I think I can workaround this for my purpose.

[ufrisk]

ok, it's probably related to the thunderbolt in some way then; if it encounters unreadable memory it stops working for some reason.

There is a standard unreadable memory region between address 0xa0000 and 0x100000.

At the moment you have to work around this with -min and -max parameters; I have an update coming in which you'll be able to specify a "memory map" in a file to avoid known problematic memory regions.

I plan to publish this and quite a few other updates in the not too distant future; but unfortunately I have to await ongoing legal discussions before pushing any more updates. I hope to be able to resolve these legal discussions within the month or two´though.

[tandasat]

Sounds good with the workaround and the future improvement plan. I will close this as I do not think we can go any further.

Thank you a lot for helping me as well as maintaining this great project.