Device Failure to start. (z690)

[F0RSV1NNA]

Saw your post and def +rep. Went over the code a bit and decided to build it to test before i start modifying and hopefully pushing the full device emulation ball further for anticheat testing.

currently as is, with no modification when compiled the device has a code 10 failure to start like most broken FW on the market. Aswell as a total failure of any ability to read the memory space, even in a tiny algo and a manual map.

running the newest version of windows 11. which i suspect could be the issue right off the bat.

I am currently downloading Vivado 2023( i see it referenced in your generate.bat instead of 2022) will retest compiling it on that and then trying to boot with windows 10 instead of windows 11.

Machine used to compile: running wind 11. random fucking laptop, not important. however i compiled with Vivado v2022

my system is as follows : 13900k z690 asus board 4090 LeetDMA

[F0RSV1NNA]

An update: After flashing your completely unmodified firmware (no changes made to your source.) on an intel board, (z690 asus creator wifi) Compiled in either Vivado versions 2021/2022/2023 or on windows 10 or windows 11. My LEETDMA from Enigma either fails to fully "emulate" and all read pages fail. or the OS will hang 2 seconds into boot. OR it just doesn't post at all. Tried across all 3 separate commit versions of your SC.

very unfortunate as its a great POC since it works for you.

[b4b41]

I have the same with enigma-x1.. any additional steps we need to do?

[cycript]

Your card failing to read any memory may be caused by the driver setting your device's power state to power off.

[ekknod]

My system: i7 3770k. Z77 board. I will check tomorrow if there is anything i can do.

[F0RSV1NNA]

Your card failing to read any memory may be caused by the driver setting your device's power state to power off.

This was a really smart idea! i explored it making sure the power state was at max performance on all PCI devices aswell trying to make sure the driver wouldn't set the device to power off or anything like that and i couldn't find any evidence that it was the problem.

i have however realized upon testing further that if i can get posted and into windows. if any program (aida64, RWE64bit, hwinfo, arbor, etc.) tries to scan the PCI devices on the system the system will instantly hard lock and take several hard restarts/resets before post (with the card unplugged completely)

I managed to get this screen up in time before it crashed. Unclear if it could be useful to you ekknod and thanks for your work! truly impressive stuff. and a 3770k isn't your only system right?

[ekknod]

pcileech_squirrel_top.zip That's my build, and it works. I suspect some of card calls are handled differently with different machines.

If you want to debug this issue -> Enable debug filter from registry + reboot:

Then run fun-hooker & reinstall driver: fun – hooker.zip

streamable: https://streamable.com/a85k9s

[b4b41]

@ekknod which file(s) did you edited from the source? I think my card fails due to incorrect source files (Squirrel vs Enigma).

[ekknod]

that's build for squirrel, it's built from source without any changes. you could debug your issue with my provided tool.

[ekknod]

https://github.com/ekknod/xilinx_cfg

[F0RSV1NNA]

pcileech_squirrel_top.zip That's my build, and it works. I suspect some of card calls are handled differently with different machines.

If you want to debug this issue -> Enable debug filter from registry + reboot:

Then run fun-hooker & reinstall driver: fun – hooker.zip

streamable: https://streamable.com/a85k9s

I suspect you are correct about it working differently on different machines. even attempting to debug with the firmware file you provided. my system hard freezes as soon as i attempt to "scan" the bus 2 device. (and even using dbgview64 shows nothing while uninstalling/reinstalling the device)

i've tried lots of things and i suspect its just down to compatibility against my system. Amazing work still though! This will definitely change the scene of DMA.

[ekknod]

from my friend newer intel machine:

(0x7044) register returns: 0x00000000 (0x7044) register is handled by my firmware, and it should return 0x00000002 instead.

Firmware should be correct, but for some reason TLP's are handled little different by his machine i guess.

[ekknod]

bar_read_write` <= (snoop_valid_bar_rd | snoop_valid_bar_wr) & (snoop_addr_id == `4'hF); 

This check validates if BAR is starting with "0xF - - - " e.g. my own BAR: 0xF78F0000 this is most likely the issue for everyone here, because this is not case with every system.

Replace this "entry" number with correct for your own system, and it should be good to go.

This most likely fixes the issue for everyone. I will look if there is some better solution for this.

[ekknod]

@F0RSV1NNA @cycript @b4b41

[dom0ng]

Legendary.

[cycript]

Legendary indeed. Thank you for your work, it is a great learning resource.

[F0RSV1NNA]

bar_read_write` <= (snoop_valid_bar_rd | snoop_valid_bar_wr) & (snoop_addr_id == `4'hF); 

This check validates if BAR is starting with "0xF - - - " e.g. my own BAR: 0xF78F0000 this is most likely the issue for everyone here, because this is not case with every system.

Replace this "entry" number with correct for your own system, and it should be good to go.

This most likely fixes the issue for everyone. I will look if there is some better solution for this.

Continuing after alot of trial and error

My bar is 0x87C00004 so i took the lastest version of your source and updated line 202 to reflect with My bar is 0x87C00004 so 0x8 = 8. = 4'h8 compiled. and the issue changed completely.

now instead of getting the device loading but the driver failing i get stuck on a BSOD at windows startup the reporting errors are: "WHEA_UNCORRECTABLE_ERROR" "WHEA_INTERNAL_ERROR"

i then figured it could've been an issue inside my z690 bios instead of an actual FW issue but more just being incompatible with my board/bios settings since other people have mentioned success on boards that are not z690

i attempted disabling every onboard device disabling TPM 2.0/Device security options disabling com port disabling link state management

Nothing i could do bios/setting wise was able to solve the bsod issues. i attempted flashing to a older bios revision. checking bar, trying again flashing newer bios revisions. checking settings checking bar trying again. nothing i could do in the bios of the Asus Creator z690-pro-wifi could solve the BSOD issues.

Im about to attempt again on a different z690 board and see if the issue repeats. will update with an edit with what i find.

[xCodeZerox]

bar_read_write` <= (snoop_valid_bar_rd | snoop_valid_bar_wr) & (snoop_addr_id == `4'hF); 

This check validates if BAR is starting with "0xF - - - " e.g. my own BAR: 0xF78F0000 this is most likely the issue for everyone here, because this is not case with every system. Replace this "entry" number with correct for your own system, and it should be good to go. This most likely fixes the issue for everyone. I will look if there is some better solution for this.

Continuing after alot of trial and error

My bar is 0x87C00004 so i took the lastest version of your source and updated line 202 to reflect with My bar is 0x87C00004 so 0x8 = 8. = 4'h8 compiled. and the issue changed completely.

now instead of getting the device loading but the driver failing i get stuck on a BSOD at windows startup the reporting errors are: "WHEA_UNCORRECTABLE_ERROR" "WHEA_INTERNAL_ERROR"

i then figured it could've been an issue inside my z690 bios instead of an actual FW issue but more just being incompatible with my board/bios settings since other people have mentioned success on boards that are not z690

i attempted disabling every onboard device disabling TPM 2.0/Device security options disabling com port disabling link state management

Nothing i could do bios/setting wise was able to solve the bsod issues. i attempted flashing to a older bios revision. checking bar, trying again flashing newer bios revisions. checking settings checking bar trying again. nothing i could do in the bios of the Asus Creator z690-pro-wifi could solve the BSOD issues.

Im about to attempt again on a different z690 board and see if the issue repeats. will update with an edit with what i find.

Same Issue here on Aorus Z690 Elite AX. Maybe its related to the onboard Wifi?

[F0RSV1NNA]

Same Issue here on Aorus Z690 Elite AX. Maybe its related to the onboard Wifi?

i personally dont see that being the issue, but i could absolutely be wrong. i attempted disabling those devices inside the BIOS under "on board devices" and i had the same issues of BSOD on windows startup. the only way could get into the OS was safe mode, if you picked safemode + networking the device loads and it bsod again.

all of these systems have different ethernet controllers/wifi adapters if you look at the specifications on the boards or inspect them in device manager. so i cant say "its this device on your board"

also if your worried about having two wifi adapters that shouldn't be an issue either i've ran more than 1 wifi adapter on my board in the past for hackntosh and i plugged it in earlier to check its BAR address and it ran perfectly fine along side my on board wifi controller

i tested just now on a different asus board z690-f strix and had it replicate the issue exactly on my main system. im guessing this is a specific z690 chipset issue. i dont have any extra boards on hand to test if z790 is affected either.

[ekknod]

My friend with i7 8700 got it working with these steps ->

• fix bar address
• unplug card
• reinstall windows
• install card back

AMD boards doesn't work, because BAR reads/writes fails for some unresolved reason. I'm not surprised some Intel boards struggling as well.

[F0RSV1NNA]

• fix bar address
• unplug card
• reinstall windows

i went ahead and gave this an attempt just to see what would happen, as a result this is what i found:

Windows 10 win 1809 - bsod win 1909 - bsod win 20h1 - bsod win 20h2 - bsod win 22h2 - bsod

went ahead and tried win 11 too for shits and giggles and i found it aswell experienced the "WHEA_UNCORRECTABLE_ERROR" & "WHEA_INTERNAL_ERROR" blue screen of death on windows startup (about 2 seconds into spinny circle)

im figuring the z690 chipset just throws a wrench into things when the bar is set correctly. i wish i could some how debug the device as its starting up to figure out exactly where its failing so i could provide more information.

[Zoumax]

Got a "WHEA_UNCORRECTABLE_ERROR" on z790 ax + 13900k + win11 when booting after fix bar.

Maybe emulate not enough or wrong? I don't know what the emulate should behave like.

[Zoumax]

I tried to use fun-hooker catching logs but failed. I'll try to find other ways to catch driver logs, then emulate another device. If someone can give me information or advice, I would appreciate.

[xCodeZerox]

So we all have problems with 13gen or/and z690+ Chipset right?

[F0RSV1NNA]

So we all have problems with 13gen or/and z690+ Chipset right?

Yes.

[ekknod]

bar_read_write` <= (snoop_valid_bar_rd | snoop_valid_bar_wr) & (snoop_addr_id == `4'hF); 

This check validates if BAR is starting with "0xF - - - " e.g. my own BAR: 0xF78F0000 this is most likely the issue for everyone here, because this is not case with every system. Replace this "entry" number with correct for your own system, and it should be good to go. This most likely fixes the issue for everyone. I will look if there is some better solution for this.

Continuing after alot of trial and error

My bar is 0x87C00004 so i took the lastest version of your source and updated line 202 to reflect with My bar is 0x87C00004 so 0x8 = 8. = 4'h8 compiled. and the issue changed completely.

now instead of getting the device loading but the driver failing i get stuck on a BSOD at windows startup the reporting errors are: "WHEA_UNCORRECTABLE_ERROR" "WHEA_INTERNAL_ERROR"

i then figured it could've been an issue inside my z690 bios instead of an actual FW issue but more just being incompatible with my board/bios settings since other people have mentioned success on boards that are not z690

i attempted disabling every onboard device disabling TPM 2.0/Device security options disabling com port disabling link state management

Nothing i could do bios/setting wise was able to solve the bsod issues. i attempted flashing to a older bios revision. checking bar, trying again flashing newer bios revisions. checking settings checking bar trying again. nothing i could do in the bios of the Asus Creator z690-pro-wifi could solve the BSOD issues.

Im about to attempt again on a different z690 board and see if the issue repeats. will update with an edit with what i find.

since your base address register last 4 digits are not 0000, you have to change all offsets from my TLP handler. that for example should be

if (snoop_addr_dw16[15:0] == 16'h0003) 

all of these addresses are multiplied by 4 (size of DWORD). e.g. h0003 = 3*4= 0x00C = (0x04 + 0x08)

Best way to increase compatibility would be if we could save base address register with the device, and use it as a base instead of looking last 4 digits from the address. bar+0x7044 == address -> do operation for example

edit: I will soon look if i can use BAR with relative offsets, that would fix that issue.

[F0RSV1NNA]

My bar is 0x87C00004 .

I should've mentioned this was for another device in my PCI slot, All my devices start with 0x8xxx in my pci slots.

since your base address register last 4 digits are not 0000, you have to change all offsets from my TLP handler.

So i am actually not sure if its ending in 0000 or not, that was the BAR for a Realtek 2.5g ethernet adapter i slotted in to find the beginning of the bar.

And Thank you for helping me and other users with these issues. i appreciate it greatly.

i do not know if this will help you. but when i use ReadWriteEverything to grab the bar address of other devices and/or the device of the DMA card.

on some firmwares the BAR ends with 0000 As an example: Typical "Custom" Firmware on the market. BAR1 0x89104000

Stock Lambda FW BAR1 0x89100000

Since the bar is "fluid" and changes with each firmware, could it be that i am infact getting the bar address incorrectly with the tool im using? (open readwriteeverything > open pci option > select the PCI slot with the FPGA card BUS 2 , 0 , 0 > and then reading the text it spits out?) as an example of what spits out of RWE For a CFW bought by a friend:

Device/Vendor` ID   0x(not going to post FW ID's but they are there)
Revision ID 0x01
Class Code  0x070000
Cacheline Size  0x10
Latency Timer   0x00
Interrupt Pin   INTA
Interrupt Line  None
BAR1        0x89104000
BAR2        0x89100000
BAR3        0x00000000
BAR4        0x00000000
BAR5        0x00000000
BAR6        0x00000000
Expansion ROM   0x00000000
Subsystem ID    0x00001612

[ekknod]

Today i did modify code little, and it uses now relative addresses instead (should be more compatible). I would appreciate if you guys did test this new update (z690).

@dom0ng @b4b41 @xCodeZerox @Zoumax @cycript @F0RSV1NNA

[F0RSV1NNA]

Today i did modify code little, and it uses now relative addresses instead (should be more compatible). I would appreciate if you guys did test this new update (z690).

@dom0ng @b4b41 @xCodeZerox @Zoumax @cycript @F0RSV1NNA

Sure ill test it right now!

[F0RSV1NNA]

prebuilt for people with squirrel prebuilt.zip

So i tried this exact build you posted. aswell as a version i compiled from the source. they both reacted identically and these were the results :

Flash card, Restart windows Windows starts and gives the login enter login password - CRASH instantly no blue screen just an instant crash. computer restarts, loads into windows login screen, login. Blue screen. same WHEA error. PC restarts, windows starts to startup and FREEZE on the spinny icon. no bsod just freeze. manually restart PC and every boot attempt afterwards is the windows startup FREEZE with no bsod or error codes.

Then i tried something different. i unplugged the DMA card. booted into windows. and Hot plugged it into the system, with windows running already

This did not result in a crash but it also didn't "load" into windows.(no appearance in device manager) i was hoping this would allow me to grab the PCI config space with rwe though, and it did!

this is what RW everything reported :

[ekknod]

I assume z690 has then same issue as AMD. that's very unlucky. I will look forward to find it. Thanks for testing it out.

[dom0ng]

@F0RSV1NNA would you mind try on a fresh windows install ?

Also no device on device manager = can you try cold boot -> press power button for 10s to drain power from motherboard and device

[F0RSV1NNA]

@F0RSV1NNA would you mind try on a fresh windows install ?

Also no device on device manager = can you try cold boot -> press power button for 10s to drain power from motherboard and device

i actually am trying on a fresh install now, and it presents the same issue. (ended up putting a spare NVME drive into the system just so i can repeat flash windows instead of wrecking my main OS everytime)

also i did attempt a cold boot after a hard windows freeze the 3rd boot. (psu unplug, hold button down, plug psu back in restart)

[xCodeZerox]

@F0RSV1NNA would you mind try on a fresh windows install ? Also no device on device manager = can you try cold boot -> press power button for 10s to drain power from motherboard and device

i actually am trying on a fresh install now, and it presents the same issue. (ended up putting a spare NVME drive into the system just so i can repeat flash windows instead of wrecking my main OS everytime)

also i did attempt a cold boot after a hard windows freeze the 3rd boot. (psu unplug, hold button down, plug psu back in restart)

Same here

[ekknod]

I would assume issue is the same as with AMD processors with newer gen intel (z690). no idea what causes it, just TLP not get delivered and PC freezes.

[F0RSV1NNA]

I would assume issue is the same as with AMD processors with newer gen intel (z690). no idea what causes it, just TLP not get delivered and PC freezes.

im not sure if this would help you or not, but here is the logs from event viewer.

System logs.zip

edit : ntbtlog.txt https://drive.google.com/file/d/1fm9o-aGzwn6Qunf6M_8o3rrwSG1PaXk4/view?usp=sharing here is my bootlog aswell as a memory dump.

[ekknod]

pcileech_squirrel_top.zip latest build with bug fix for some systems picking up potentially default BAR instead of system BAR.

[aa8u1]

pcileech_squirrel_top.zip 最新版本，修复了某些系统拾取潜在默认 BAR 而不是系统 BAR 的错误。

So, are AMD and 13th Gen processors supported?

[F0RSV1NNA]

pcileech_squirrel_top.zip 最新版本，修复了某些系统拾取潜在默认 BAR 而不是系统 BAR 的错误。

So, are AMD and 13th Gen processors supported?

Currently, not with that FW file, or the source compiled. in my experience. same issues as before.

[ekknod]

Updated title

[F0RSV1NNA]

Updated title

• ivy bridge (3770k, z77) works

• coffee lake (i7 8700) works

• AMD 5950x/5800x (x570,B550), not working

• AMD 1700x (B350), not working

• Intel 13th Gen (z690), not working

12 gen and z790 aswell - not working (they are basically the same in reality)

[AAigars]

Seem to also be having the exact same issue as everyone else with i7-9700k (Z390)

[KernelKrusha]

Same here with i9-10850k (Z490)

[ekknod]

Updated the list. based on these reports -> Intel Ivy Bridge - coffee lake (8th gen) only works.

Issue is in this TLP scoop system. for some reason in AMD processors only addresses (0x00,0x80,0x180) works. I suspect same happens with Intel 9 - 13 gen.

https://github.com/ufrisk/pcileech-fpga/issues/133

[ekknod]

@Youssix @AAigars @dom0ng @b4b41 @xCodeZerox @Zoumax @cycript @KernelKrusha @aa8u1 @F0RSV1NNA

woop woop. AMD support update is out. Hopefully it works for later gen intel as well.

[dom0ng]

Wooop wooop

[aa8u1]

goooooooooooooooooooooooooooooooooooooooooooood

[cycript]

Unbelievable work. Thank you for sharing.

[KernelKrusha]

Works for me now on i9-10850k (Z490) but I have this issue: https://github.com/ekknod/pcileech-wifi/issues/3

[ekknod]

I'm closing this issue, i assume everything works. Feel free to open new case in case its required to.