search

Xilinx / XRT

Run Time for AIE and FPGA based platforms
https://xilinx.github.io/XRT
Other
549 stars 464 forks source link

SecureBoot and XRT doesn't seem to mix properly #6813

Open quantgeek opened 2 years ago

quantgeek commented 2 years ago

It seems that when SecureBoot is enabled, the U250 and XRT doesn't work. There seems to be some code to handle it, but it just doesn't seem to work. What is it supposed to do as there is some code that handles signing? See:

$ ./HelloWorld vadd.xclbin Found Platform Platform Name: Xilinx INFO: Reading vadd.xclbin Loading: 'vadd.xclbin' Trying to program device[0]: xilinx_u250_gen3x16_xdma_shell_4_1 XRT build version: 2.13.466 Build hash: f5505e402c2ca1ffe45eb6d3a9399b23a0dc8776 Build date: 2022-04-14 17:43:11 Git branch: 2022.1 PID: 22667 UID: 1000 [Mon Jun 27 21:37:58 2022 GMT] HOST: Gilgamesh EXE: /home/quantgeek/Documents/FPGA/Xilinx/workspace/HelloWorld/Hardware/HelloWorld [XRT] ERROR: Xclbin isn't signed properly [XRT] ERROR: See dmesg log for details. err = -129 [XRT] ERROR: failed to load xclbin: Key was rejected by service Failed to program device[0] with xclbin file! Failed to program any device found, exit!

Looking at dmesg indicates:

[ 5808.488229] xocl 0000:0b:00.1: icap.u.22020096 ffff920e83e7ec10 icap_download_bitstream_axlf: incoming xclbin: 40fbd00b-5937-5b71-505c-1f82bb8759cd on device xclbin: 00000000-0000-0000-0000-000000000000 [ 5808.490331] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_kds_xgq_cfg_start: Config start completed, num_cus(2493), num_scus(-696781782) [ 5808.638806] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_kds_xgq_cfg_end: Config end completed [ 5808.638832] xocl 0000:0b:00.1: mailbox.u.9437184 ffff920c357fb010 _mailbox_request: sending request: 7 via HW [ 5808.639080] xclmgmt 0000:0b:00.0: mailbox.m.9437184 ffff921bda3b4010 process_request: received request from peer: 7, passed on [ 5808.658950] xclmgmt 0000:0b:00.0: icap.m.22020096 ffff921bd0a08c10 icap_download_bitstream_axlf: check interface uuid [ 5808.658967] xclmgmt 0000:0b:00.0: icap.m.22020096 ffff921bd0a08c10 icap_download_bitstream_axlf: incoming xclbin: 40fbd00b-5937-5b71-505c-1f82bb8759cd on device xclbin: 00000000-0000-0000-0000-000000000000 [ 5808.658974] xclmgmt 0000:0b:00.0: ffff920c083a00c8 raptor_cmc_access: Release CMC succeeded. [ 5808.659000] xclmgmt 0000:0b:00.0: icap.m.22020096 ffff921bd0a08c10 icap_cache_bitstream_axlf_section: found kind 6(MEM_TOPOLOGY) [ 5808.659005] xclmgmt 0000:0b:00.0: icap.m.22020096 ffff921bd0a08c10 icap_cache_bitstream_axlf_section: found kind 8(IP_LAYOUT) [ 5808.659008] xclmgmt 0000:0b:00.0: icap.m.22020096 ffff921bd0a08c10 icap_verify_signed_signature: xclbin is not signed, rejected **[ 5808.659012] xclmgmt 0000:0b:00.0: icap.m.22020096 ffff921bd0a08c10 icap_xclbin_download: ret: -129 [ 5808.659015] xclmgmt 0000:0b:00.0: icap.m.22020096 ffff921bd0a08c10 icap_download_bitstream_axlf: err: -129** [ 5808.659655] xclmgmt 0000:0b:00.0: mailbox.m.9437184 ffff921bda3b4010 mailbox_post_response: posting response for: 7 via HW [ 5808.659847] xocl 0000:0b:00.1: icap.u.22020096 ffff920e83e7ec10 icap_peer_xclbin_download: peer xclbin download err: -129 [ 5808.659869] xocl 0000:0b:00.1: icap.u.22020096 ffff920e83e7ec10 icap_cache_bitstream_axlf_section: found kind 6(MEM_TOPOLOGY) [ 5808.659883] xocl 0000:0b:00.1: icap.u.22020096 ffff920e83e7ec10 icap_cache_bitstream_axlf_section: found kind 26(ASK_GROUP_TOPOLOGY) [ 5808.659888] xocl 0000:0b:00.1: icap.u.22020096 ffff920e83e7ec10 icap_download_bitstream_axlf: err: -129 [ 5808.659892] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_init_mem: Topology count = 9, data_length = 360 [ 5808.659896] xocl 0000:0b:00.1: p2p.u.11534336 ffff920e83e7c010 p2p_mem_init: already initialized [ 5808.659912] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_init_mem: Memory Bank: bank0 [ 5808.659914] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_init_mem: Base Address:0x4000000000 [ 5808.659917] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_init_mem: Size:0x400000000 [ 5808.659931] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_init_mem: Type:1 [ 5808.659933] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_init_mem: Used:0 [ 5808.659935] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_init_mem: Memory Bank: bank1 [ 5808.659938] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_init_mem: Base Address:0x5000000000 [ 5808.659940] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_init_mem: Size:0x400000000 [ 5808.659942] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_init_mem: Type:1 [ 5808.659943] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_init_mem: Used:1 [ 5808.659947] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_init_mem: Memory Bank: bank2 [ 5808.659948] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_init_mem: Base Address:0x6000000000 [ 5808.659950] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_init_mem: Size:0x400000000 [ 5808.659952] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_init_mem: Type:1 [ 5808.659953] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_init_mem: Used:0 [ 5808.659955] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_init_mem: Memory Bank: bank3 [ 5808.659956] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_init_mem: Base Address:0x7000000000 [ 5808.659958] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_init_mem: Size:0x400000000 [ 5808.659960] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_init_mem: Type:2 [ 5808.659961] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_init_mem: Used:0 [ 5808.659963] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_init_mem: Memory Bank: PLRAM[0] [ 5808.659964] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_init_mem: Base Address:0x3000000000 [ 5808.659966] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_init_mem: Size:0x20000 [ 5808.659968] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_init_mem: Type:2 [ 5808.659970] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_init_mem: Used:0 [ 5808.659971] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_init_mem: Memory Bank: PLRAM[1] [ 5808.659973] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_init_mem: Base Address:0x3000200000 [ 5808.659975] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_init_mem: Size:0x20000 [ 5808.659977] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_init_mem: Type:2 [ 5808.659978] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_init_mem: Used:0 [ 5808.659980] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_init_mem: Memory Bank: PLRAM[2] [ 5808.659982] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_init_mem: Base Address:0x3000400000 [ 5808.659983] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_init_mem: Size:0x20000 [ 5808.659985] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_init_mem: Type:2 [ 5808.659987] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_init_mem: Used:0 [ 5808.659989] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_init_mem: Memory Bank: PLRAM[3] [ 5808.659991] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_init_mem: Base Address:0x3000600000 [ 5808.659992] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_init_mem: Size:0x20000 [ 5808.659994] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_init_mem: Type:2 [ 5808.659996] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_init_mem: Used:0 [ 5808.659998] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_init_mem: Memory Bank: HOST[0] [ 5808.660000] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_init_mem: Base Address:0x2000000000 [ 5808.660002] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_init_mem: Size:0x400000000 [ 5808.660003] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_init_mem: Type:2 [ 5808.660005] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_init_mem: Used:1 [ 5808.660010] xocl 0000:0b:00.1: p2p.u.11534336 ffff920e83e7c010 p2p_mem_map: map bank addr 0x5000000000, size 17179869184, offset 0, len 0 [ 5808.660013] xocl 0000:0b:00.1: p2p.u.11534336 ffff920e83e7c010 p2p_bar_map: bank addr 5000000000, sz 17179869184, slots 256 [ 5808.660016] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_init_mem: Allocating Memory Bank: bank1 [ 5808.660018] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_init_mem: base_addr:0x5000000000, total size:0x400000000 [ 5808.660020] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_init_mem: Found a new memory region [ 5808.660023] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_init_mem: Allocating Memory Bank: HOST[0] [ 5808.660025] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_init_mem: base_addr:0x2000000000, total size:0x400000000 [ 5808.660027] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_init_mem: Found a new memory region [ 5808.660030] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_set_cma_bank: Could not find reserved HOST mem, Skipped [ 5808.660033] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_init_mem: drm_mm_init called for the available memory range [ 5808.660036] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_init_mem: ret 0 [ 5808.660037] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_read_axlf_helper: Failed to download xclbin, err: -129 [ 5808.721995] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_destroy_client: client exits pid(22667) [ 5808.722003] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_drvinst_close: CLOSE 2 [ 5808.722006] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_drvinst_close: NOTIFY 0000000029712d2b [ 5951.733336] xocl 0000:0b:00.1: ffff920c083a60c8 _xocl_drvinst_open: OPEN 1 [ 5951.733508] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_create_client: created KDS client for pid(22871), ret: 0 [ 5951.742510] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_destroy_client: client exits pid(22871) [ 5951.742519] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_drvinst_close: CLOSE 2 [ 5951.742522] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_drvinst_close: NOTIFY 0000000029712d2b [ 5951.989847] xocl 0000:0b:00.1: ffff920c083a60c8 _xocl_drvinst_open: OPEN 1 [ 5951.989889] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_create_client: created KDS client for pid(22996), ret: 0 [ 5952.001150] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_destroy_client: client exits pid(22996) [ 5952.001164] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_drvinst_close: CLOSE 2 [ 5952.001167] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_drvinst_close: NOTIFY 0000000029712d2b [ 5952.040497] xocl 0000:0b:00.1: ffff920c083a60c8 _xocl_drvinst_open: OPEN 1 [ 5952.040547] xocl 0000:0b:00.1: ffff920c083a60c8 xocl_create_client: created KDS client for pid(23114), ret: 0

uday610 commented 2 years ago

This error is by design, we require signed XCLBIN after secure boot enabled

quantgeek commented 2 years ago

Well, I tried to sign both HelloWorld and vadd.xclbin before I posted, but that didn't work either. The system was already signed before I installed XRT and the U250 board, so that isn't it, unless I am missing something. What exactly needs to be signed and I will try again.

maxzhen commented 2 years ago

You've made sure the key you used to sign them was also known to the Linux kernel, so that they are available for verification?

quantgeek commented 2 years ago

Yes I did. The keys are the same one I use EVERYWHERE. In fact, I can't load the Xilinx kernel drivers without signing them and they are all signed and loaded.

maxzhen commented 2 years ago

This is good, then. From the log msg and what you've described here, it seems that the xclbin was not signed properly. @rozumx may be able to help to verify the signing process.

rozumx commented 2 years ago

Q: Have you looked at the regression tests regarding how the xclbin containers are signed?

These tests:

  1. Creates the keys (e.g., CER or DER)
  2. Signs the xclbin containers
  3. Validate the signatures against CER and DER formatted keys
quantgeek commented 2 years ago

I wasn't aware of these test scripts. Let me try these out and see what is causing the problem. Thanks!