Advice on how to debug when there is no connection

[OFTK]

Hi

I am using the ac701/ft601 platform, and it seems I have successfully flashed the device. I am running the toolkit on a windows pc.

When trying to run any pcileeh command it just returns - “PCILEECH: Failed to connect to the device”.

Any idea how I can go on and debug the system?

Thanks!

[ufrisk]

1) make sure you have the required FTDI driver as described in the guide: https://github.com/ufrisk/pcileech/wiki/PCILeech-on-Windows

2) If pressing the lowest button in the lower right corner a LED will blink in the upper right corner of the AC701 if successfully flashed.

3) Check out the https://github.com/ufrisk/LeechCore/wiki/Device_FPGA

4) connect with pcileech.exe -v -vv -device fpga display -min 0x1000 this will show if you are able to communicate with the AC701 FPGA over USB (you'll see the bitstream version of the fpga in the output). If you are able to communicate it will also show if PCIe is working (PCIe id is shown, four hex-digits in output - non zero if working)

5) check out issues in this repository and also in the https://github.com/ufrisk/pcileech-fpga repository

Please let me know if you manage to get it working or if you do have more questions.

[OFTK]

Hi Thank you for the quick answer!

When writing the command:

pcileech.exe -v -vv -device fpga display -min 0x1000

I get: [2, v3.3, 0000]. So I guess the problem is with the PCI core setup. What could be the problem here? I’m no expert in hardware implemented PCI cores, so could the problem be with the PCI interface? Something about the target pc? (I took a normal pc, opened it, and connected the AC701 pci to a free PCI slot I saw)

On 12 Jul 2019, at 18:53, Ulf Frisk notifications@github.com wrote:

make sure you have the required FTDI driver as described in the guide: https://github.com/ufrisk/pcileech/wiki/PCILeech-on-Windows

If pressing the lowest button in the lower right corner a LED will blink in the upper right corner of the AC701 if successfully flashed.

Check out the https://github.com/ufrisk/LeechCore/wiki/Device_FPGA

connect with pcileech.exe -v -vv -device fpga display -min 0x1000 this will show if you are able to communicate with the AC701 FPGA over USB (you'll see the bitstream version of the fpga in the output). If you are able to communicate it will also show if PCIe is working (PCIe id is shown, four hex-digits in output - non zero if working)

check out issues in this repository and also in the https://github.com/ufrisk/pcileech-fpga repository

Please let me know if you manage to get it working or if you do have more questions.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[ufrisk]

Yes, [2, v3.3, 0000] indicates that there is no pcie id assigned to the core by the target system, but that the usb<->fpga communication is up and working. You can try to insert the fpga in another slot, and power cycle the fpga and reboot the target system and hopefully it will start working.

[OFTK]

Thank you it works!

[eljeffeg]

I'm having a similar problem with the ac701_ft601. I've tried version 3.2 & 3.4 and both tell me 0000. I've tried plugging the Xilinx card directly into the PCI slot, tried various slots. Double checked the jumpers on the ft601. @ufrisk Is there something I'm missing or anything I can do to test? I just got the card in and was hoping to get it running so I can have some experience before I camp at your Defcon demo and pick your brain.

[ufrisk]

@jeffg2k if you get the bitstream version number in the debug output the FT601 connection is fine.

If you do have bitstream version numbers and 0000 at the end in the debug output that means there is something wrong with the PCIe connection. Try another slot, reboot the board (power cycle), reboot the computer and so on ... I hope you get it to work before my DEFCON demo, otherwise please stop by and ask me about it afterwards :)

[eljeffeg]

I'm still having a lot of problems with this. It's the PCIe connection piece as I get [2, v3.4, 0000]. I've only gotten it to communicate the PCI on a Linux machine but didn't get the kernel implant to work. Tested two Windows machines. A Dell laptop and a Dell workstation. I turned off the bios virtualization support on each. On the laptop, I tried via Mini-PCI via the Wifi card, no luck. Tried via a PCMIA card (since it had a port) and no luck. Tried on the desktop via direct PCI with the same [2, v3.4, 0000].

Some things to note - I'm running the Windows 7 x64 host system in a Virtual Machine (usb3 specified) - VMWare Fusion.

[ufrisk]

The USB connection between FPGA / attacker computer is working fine. The problem is the PCIe connection.

When you do get a PCIe link the LED marked "GPIO LEDS 1" in the upper right corner will light up.

Sometimes it helps having both computer / FPGA powered down, then power on the FPGA, then power on the computer shortly afterwards.

[eljeffeg]

I'm not getting a light on GPIO LEDS 1.

[eljeffeg]

Here is another system where I've plugged the card directly. I've tried the procedure of "having both computer / FPGA powered down, then power on the FPGA, then power on the computer shortly afterwards." The GPIO LEDS 1 light never comes on. I've disabled any virtualization in the bios.

[ufrisk]

1) what happens if you press SW5 (located at the bottom right?) - does GPIO LEDS 0 light up?

2) what is the output if you run pcileech display -device fpga -v

[eljeffeg]

@ufrisk when I press SW5, GPIO LEDS 0 lights up as the button is pressed and goes out when the button is released. The output of pcileech display -device fpga -v is:

DEVICE: FPGA: ERROR: Unable to retrieve required Device PCIe ID [2,v3.4,0000]
PCILEECH: Failed to connect to the device.

Here are some close up images. Perhaps you'll see something (maybe a jumper or switch) that is incorrect. I'm trying to read through the AC701 Eval Kit - maybe I can monitor what's going on in some way.

[ufrisk]

There is a problem with the PCIe link to your PC.

Can you please power down both the board and the PC. Then power on the board. Then power on the PC.

Also please try different PCIe slots, and if you have the ability, try different computers, try (if possible) inserting the AC701 directly into the PCIe slot without using the extender cable.

Unfortunately it's very hard for me to tell anything but this, there aren't really anything more I can do than this.

[eljeffeg]

Unfortunately I've already tried the power trick, another slot, another computer, and inserting directly. I've tried 3 computers (1 works, but is Linux and won't load the kernel module) - the other two (Windows) do not get the PCIe link. Is this typical or should I be asking for a refund from XILINX (or going to their support)?

[eljeffeg]

Looks like someone else had a similar problem: https://forums.xilinx.com/t5/PCI-Express/AC701-PCIe-link-is-never-UP/m-p/985937 So I can try to go through this process, make sure the card is working with all their default software and see what shakes out. Thanks for your help @ufrisk - you're awesome and I know looking at these support requests sucks.

[eljeffeg]

@ufrisk Wanted to give you an update. The Xilinx XTP227 - AC701 PCIe Tutorial (v11.0) appears to work and I see the appropriate pcie link light in GPIO.

When I load up the pcileech prebuilt bitstream or build the project myself, I no longer get the pcie link. I'm using the latest Vivado v2019.1.3

[ufrisk]

so I guess there must be some kind of issue with my bitstream then.

problem is that I have no way of replicating this issue, it works for me with the pre-build bitstream and I also use Vivado 2019.1.3 to flash ...

[eljeffeg]

I was wondering if it may be a setting, such as 1x vs 2x vs 4x, 2.5gb vs 5gb, or some other internal that might offer more compatibility. I think the sample project uses 4x, while pcileech uses 2x. I've reviewed the device id and I think it's the same as the PCIE sample project, so that shouldn't be the problem. Not sure how best to compare how pcileech and the pcie sample project differ in how they get the pci link. There is an option to enable jtag debugging. Not sure what it would allow us to see, but maybe something could be added to give greater visibility into the pcie negotiation process and where it is getting hung up. I'm trying to learn Vivado - fpga dev is new to me, but I'm willing to be your remote hands to get it working, but I realize this is a lot of support for free software. If you think it would help, I could probably ship you my card (covering round trip) if we think it might be the card itself. It passes the BIST.

[ufrisk]

Thanks for the offer. I'm pretty sure it's something in my design and how it interacts with your system. It's not the x2 vs x4 setting, since that cable you use is only x1 anyway - so the link will downgrade to x1 when that cable is used...

Also in the current design I use the clock from the FT601 daughter board for some internal things, so it may be a good idea to have that board connected (no need to have the usb connected) while running the tests.

If you're willing to try out a few changes and see how it works:

1. change around a few values in the PCIe block, just double click on it in the design. Do not change the major things that will break the design, such as clock rates or x2. The most interesting values are the TLP "Maximum Payload Size" - which is currently set to 128 bytes. Can you try to change this to a larger values - even 1024 bytes? regenerate the core and rebuild the bitstream. Also try to change the other values to make the core as similar as your sample core as possible.

[eljeffeg]

Turns out, the base_tdr puts the PCI link (user_lnk_up) on GPIO 0, pcie sample puts it on GPIO 2, and PCILeech puts it on GPIO 1. So, I actually haven't gotten the PCIe link on any of them as I was looking at the wrong led in the other cases. I'll go back to trying to get them to work reliably first and then revisit. Thanks for your help @ufrisk

[ufrisk]

Thanks for the update, please let me know if there is any progress. Good Luck :)

[ufrisk]

I'm closing this issue due to old age. Since I haven't heard anything back I'm guessing it's resolved.

[eljeffeg]

I was never able to resolve it. Spent many days trying and even got Xilinx support, but the covid and work from home has prevented me from working on it further. Was trying to get the PCI Screamer m2 and hope that fixes it.