Proposal of io address of vc707 to match HiFive Unleashed

[mcd500]

This is just a proposal. We are trying to run Keystone on vc707 with minimal changes in the source code of Keystone which runs perfectly on HiFive Unleashed.

These are the io addresses of vc707 would make them matches with HiFive Unleashed. Would be a problem for making pull request after testing them?

diff --git a/bootrom/sdboot/include/platform.h b/bootrom/sdboot/include/platform.h
index 73052bd..2b76f70 100644
--- a/bootrom/sdboot/include/platform.h
+++ b/bootrom/sdboot/include/platform.h
@@ -36,9 +36,9 @@
 #define CLINT_CTRL_SIZE _AC(0x10000,UL)
 #define DEBUG_CTRL_ADDR _AC(0x0,UL)
 #define DEBUG_CTRL_SIZE _AC(0x1000,UL)
-#define ERROR_MEM_ADDR _AC(0x3000,UL)
+#define ERROR_MEM_ADDR _AC(0x18000000,UL)
 #define ERROR_MEM_SIZE _AC(0x1000,UL)
-#define GPIO_CTRL_ADDR _AC(0x64002000,UL)
+#define GPIO_CTRL_ADDR _AC(0x10060000,UL)
 #define GPIO_CTRL_SIZE _AC(0x1000,UL)
 #define MASKROM_MEM_ADDR _AC(0x10000,UL)
 #define MASKROM_MEM_SIZE _AC(0x2000,UL)
@@ -46,11 +46,11 @@
 #define MEMORY_MEM_SIZE _AC(0x40000000,UL)
 #define PLIC_CTRL_ADDR _AC(0xc000000,UL)
 #define PLIC_CTRL_SIZE _AC(0x4000000,UL)
-#define SPI_CTRL_ADDR _AC(0x64001000,UL)
+#define SPI_CTRL_ADDR _AC(0x10040000,UL)
 #define SPI_CTRL_SIZE _AC(0x1000,UL)
 #define TEST_CTRL_ADDR _AC(0x4000,UL)
 #define TEST_CTRL_SIZE _AC(0x1000,UL)
-#define UART_CTRL_ADDR _AC(0x64000000,UL)
+#define UART_CTRL_ADDR _AC(0x10010000,UL)
 #define UART_CTRL_SIZE _AC(0x1000,UL)

 // IOF masks
diff --git a/bootrom/sdboot/linker/memory.lds b/bootrom/sdboot/linker/memory.lds
index 7f6cfc9..cf02376 100644
--- a/bootrom/sdboot/linker/memory.lds
+++ b/bootrom/sdboot/linker/memory.lds
@@ -1,14 +1,14 @@
 MEMORY
 {
   debug_ctrl (rwx) : ORIGIN = 0x0, LENGTH = 0x1000
-  error_mem (rw) : ORIGIN = 0x3000, LENGTH = 0x1000
+  error_mem (rw) : ORIGIN = 0x18000000, LENGTH = 0x1000
   test_ctrl (rw) : ORIGIN = 0x4000, LENGTH = 0x1000
   maskrom_mem (rx) : ORIGIN = 0x10000, LENGTH = 0x2000
   clint_ctrl (rw) : ORIGIN = 0x2000000, LENGTH = 0x10000
   plic_ctrl (rw) : ORIGIN = 0xc000000, LENGTH = 0x4000000
   axi_pcie_host_1_00_a_ctrl (rw) : ORIGIN = 0x50000000, LENGTH = 0x4000000
-  uart_ctrl (rw) : ORIGIN = 0x54000000, LENGTH = 0x1000
-  spi_ctrl (rw) : ORIGIN = 0x54001000, LENGTH = 0x1000
-  gpio_ctrl (rw) : ORIGIN = 0x54002000, LENGTH = 0x1000
+  uart_ctrl (rw) : ORIGIN = 0x10010000, LENGTH = 0x1000
+  spi_ctrl (rw) : ORIGIN = 0x10040000, LENGTH = 0x1000
+  gpio_ctrl (rw) : ORIGIN = 0x10060000, LENGTH = 0x1000
   memory_mem (rwx) : ORIGIN = 0x80000000, LENGTH = 0x40000000
 }
diff --git a/fpga-shells b/fpga-shells
--- a/fpga-shells
+++ b/fpga-shells
@@ -1 +1 @@
-Subproject commit 9eb56625ba252e6abbb7ded5158331b7da7ea39a
+Subproject commit 9eb56625ba252e6abbb7ded5158331b7da7ea39a-dirty
diff --git a/src/main/scala/unleashed/DevKitConfigs.scala b/src/main/scala/unleashed/DevKitConfigs.scala
index 207c90a..3e4b4ff 100644
--- a/src/main/scala/unleashed/DevKitConfigs.scala
+++ b/src/main/scala/unleashed/DevKitConfigs.scala
@@ -24,11 +24,11 @@ class FreedomU500Config extends Config(
 // Freedom U500 Dev Kit Peripherals
 class U500DevKitPeripherals extends Config((site, here, up) => {
   case PeripheryUARTKey => List(
-    UARTParams(address = BigInt(0x64000000L)))
+    UARTParams(address = BigInt(0x10010000L)))
   case PeripherySPIKey => List(
-    SPIParams(rAddress = BigInt(0x64001000L)))
+    SPIParams(rAddress = BigInt(0x10040000L)))
   case PeripheryGPIOKey => List(
-    GPIOParams(address = BigInt(0x64002000L), width = 4))
+    GPIOParams(address = BigInt(0x10060000L), width = 4))
   case PeripheryMaskROMKey => List(
     MaskROMParams(address = 0x10000, name = "BootROM"))
 })
@@ -40,7 +40,7 @@ class U500DevKitConfig extends Config(
   new FreedomU500Config().alter((site,here,up) => {
     case SystemBusKey => up(SystemBusKey).copy(
       errorDevice = Some(DevNullParams(
-        Seq(AddressSet(0x3000, 0xfff)),
+        Seq(AddressSet(0x18000000, 0xfff)),
         maxAtomic=site(XLen)/8,
         maxTransfer=128,
         region = RegionType.TRACKED)))

[mcd500]

This is the log of Keystone running on vc707 after changing the io addresses. Not perfectly but starting to working.

INIT
CMD0
CMD8
ACMD41
CMD58
CMD16
CMD18
LOADING  
BOOT
bbl loader
secure boot
create a random seed
copy TEST device key
measure SM
generate sm_key_seed
generate public keys
endose and sign
clean up and go

                SIFIVE, INC.

         5555555555555555555555555
        5555                   5555
       5555                     5555
      5555                       5555
     5555       5555555555555555555555
    5555       555555555555555555555555
   5555                             5555
  5555                               5555
 5555                                 5555
5555555555555555555555555555          55555
 55555           555555555           55555
   55555           55555           55555
     55555           5           55555
       55555                   55555
         55555               55555
           55555           55555
             55555       55555
               55555   55555
                 555555555
                   55555
                     5

           SiFive RISC-V Core IP
miiininnintititiaialaliliziziziningng g s smsm
tinit

d iaialinilniiztziienadgl  isszmme
 i 
   tisma
zedzneitid salim
    sm
[    0.000000] OF: fdt: Ignoring memory range 0x80000000 - 0x80200000
[    0.000000] Linux version 4.15.0-00060-g65e929792fb9-dirty () (gcc version 8.2.0 (GCC)) #45 SMP Mon Apr 19
[    0.000000] bootconsole [early0] enabled
[    0.000000] Initial ramdisk at: 0x        (ptrval) (12032512 bytes)
[    0.000000] Zone ranges:
[    0.000000]   DMA32    [mem 0x0000000080200000-0x00000000bfffffff]
[    0.000000]   Normal   [mem 0x00000000c0000000-0x00000bffffffffff]
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000080200000-0x00000000bfffffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000080200000-0x00000000bfffffff]
[    0.000000] software IO TLB [mem 0xbb1fb000-0xbf1fb000] (64MB) mapped at [        (ptrval)-        (ptrval)]
[    0.000000] elf_hwcap is 0x112d
[    0.000000] percpu: Embedded 16 pages/cpu @        (ptrval) s28632 r8192 d28712 u65536
[    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 258055
[    0.000000] Kernel command line: earlyprintk
[    0.000000] Dentry cache hash table entries: 131072 (order: 8, 1048576 bytes)
[    0.000000] Inode-cache hash table entries: 65536 (order: 7, 524288 bytes)
[    0.000000] Sorting __ex_table...
[    0.000000] Memory: 947372K/1046528K available (3086K kernel code, 217K rwdata, 1049K rodata, 11911K init, 767K bss, 99156K reserved,)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
[    0.000000] Hierarchical RCU implementation.
[    0.000000]  RCU event tracing is enabled.
[    0.000000] NR_IRQS: 0, nr_irqs: 0, preallocated irqs: 0
[    0.000000] riscv,cpu_intc,0: 64 local interrupts mapped
[    0.000000] riscv,cpu_intc,1: 64 local interrupts mapped
[    0.000000] riscv,cpu_intc,2: 64 local interrupts mapped
[    0.000000] riscv,cpu_intc,3: 64 local interrupts mapped
[    0.000000] ------------[ cut here ]------------
[    0.000000] WARNING: CPU: 2 PID: 0 at /home/riscv/project/keystone/riscv-linux/drivers/irqchip/irq-riscv-intc.c:167 riscv_irq_enable+2
[    0.000000] Modules linked in:
[    0.000000] CPU: 2 PID: 0 Comm: swapper/2 Not tainted 4.15.0-00060-g65e929792fb9-dirty #45
[    0.000000] Call Trace:
[    0.000000] [<        (ptrval)>] walk_stackframe+0x0/0xa2
[    0.000000] [<        (ptrval)>] show_stack+0x26/0x34
[    0.000000] [<        (ptrval)>] dump_stack+0x5e/0x7c
[    0.000000] [<        (ptrval)>] __warn+0x98/0xd6
[    0.000000] [<        (ptrval)>] warn_slowpath_null+0x2c/0x3e
[    0.000000] [<        (ptrval)>] riscv_irq_enable+0x8c/0x92
[    0.000000] [<        (ptrval)>] irq_enable+0x28/0x5e
[    0.000000] [<        (ptrval)>] __irq_startup+0x5a/0x88
[    0.000000] [<        (ptrval)>] irq_startup+0x48/0xe6
[    0.000000] [<        (ptrval)>] irq_activate_and_startup+0x1c/0x28
[    0.000000] [<        (ptrval)>] __irq_do_set_handler+0x6e/0x132
[    0.000000] [<        (ptrval)>] irq_set_chained_handler_and_data+0x2c/0x4a
[    0.000000] [<        (ptrval)>] plic_init+0x1f6/0x322
[    0.000000] [<        (ptrval)>] of_irq_init+0x136/0x252
[    0.000000] [<        (ptrval)>] irqchip_init+0x10/0x1c
[    0.000000] [<        (ptrval)>] init_IRQ+0x8/0x14
[    0.000000] [<        (ptrval)>] start_kernel+0x20a/0x34c
[    0.000000] [<        (ptrval)>] _sinittext+0x4e/0x56
[    0.000000] random: get_random_bytes called from print_oops_end_marker+0x1e/0x3e with crng_init=0
[    0.000000] ---[ end trace 66814e3a8c80ec10 ]---
[    0.000000] riscv,plic0,c000000: mapped 6 interrupts to 4/8 handlers
[    0.000000] clocksource: riscv_clocksource: mask: 0xffffffffffffffff max_cycles: 0x1d854df40, max_idle_ns: 3526361616960 ns
[    0.000068] sched_clock: 64 bits at 1000kHz, resolution 1000ns, wraps every 2199023255500ns
[    0.008947] Calibrating delay loop (skipped), value calculated using timer frequency.. 2.00 BogoMIPS (lpj=10000)
[    0.018733] pid_max: default: 32768 minimum: 301
[    0.024889] Mount-cache hash table entries: 2048 (order: 2, 16384 bytes)
[    0.031445] Mountpoint-cache hash table entries: 2048 (order: 2, 16384 bytes)
[    0.053210] Hierarchical SRCU implementation.
[    0.064177] smp: Bringing up secondary CPUs ...
[    0.073845] clocksource: riscv_clocksource: mask: 0xffffffffffffffff max_cycles: 0x1d854df40, max_idle_ns: 3526361616960 ns
[    0.084425] sched_clock: 64 bits at 1000kHz, resolution 1000ns, wraps every 2199023255500ns
[    0.100243] clocksource: riscv_clocksource: mask: 0xffffffffffffffff max_cycles: 0x1d854df40, max_idle_ns: 3526361616960 ns
[    0.110822] sched_clock: 64 bits at 1000kHz, resolution 1000ns, wraps every 2199023255500ns
[    0.126699] clocksource: riscv_clocksource: mask: 0xffffffffffffffff max_cycles: 0x1d854df40, max_idle_ns: 3526361616960 ns
[    0.137273] sched_clock: 64 bits at 1000kHz, resolution 1000ns, wraps every 2199023255500ns
[    0.147092] smp: Brought up 1 node, 4 CPUs
[    0.156332] devtmpfs: initialized
[    0.176659] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
[    0.186193] futex hash table entries: 1024 (order: 4, 65536 bytes)
[    0.198952] NET: Registered protocol family 16
[    0.272917] vgaarb: loaded
[    0.278374] SCSI subsystem initialized
[    0.285850] usbcore: registered new interface driver usbfs
[    0.291684] usbcore: registered new interface driver hub
[    0.297266] usbcore: registered new device driver usb
[    0.303220] pps_core: LinuxPPS API ver. 1 registered
[    0.307523] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[    0.317120] PTP clock support registered
[    0.327146] clocksource: Switched to clocksource riscv_clocksource
[    0.348345] NET: Registered protocol family 2
[    0.358438] TCP established hash table entries: 8192 (order: 4, 65536 bytes)
[    0.366785] TCP bind hash table entries: 8192 (order: 5, 131072 bytes)
[    0.375551] TCP: Hash tables configured (established 8192 bind 8192)
[    0.384157] UDP hash table entries: 512 (order: 2, 16384 bytes)
[    0.390125] UDP-Lite hash table entries: 512 (order: 2, 16384 bytes)
[    0.398396] NET: Registered protocol family 1
[    1.386452] Unpacking initramfs...
[    2.451881] Initialise system trusted keyrings
[    2.457276] workingset: timestamp_bits=62 max_order=18 bucket_order=0
[    2.539834] random: fast init done
[    2.543528] Key type asymmetric registered
[    2.547361] Asymmetric key parser 'x509' registered
[    2.551959] io scheduler noop registered
[    2.558687] io scheduler cfq registered (default)
[    2.562748] io scheduler mq-deadline registered
[    2.567448] io scheduler kyber registered
[    2.580808] sifive_gpio 64002000.gpio: SiFive GPIO chip registered 4 GPIOs
[    3.200375] Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
[    3.217250] 64000000.serial: ttySI0 at MMIO 0x64000000 (irq = 1, base_baud = 0) is a sifive-serial
[    3.225753] console [ttySI0] enabled
[    3.225753] console [ttySI0] enabled
[    3.232857] bootconsole [early0] disabled
[    3.232857] bootconsole [early0] disabled
[    3.259041] sifive_spi 64001000.spi: mapped; irq=2, cs=1
[    3.270383] libphy: Fixed MDIO Bus: probed
[    3.276146] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    3.282207] ehci-pci: EHCI PCI platform driver
[    3.288252] usbcore: registered new interface driver usb-storage
[    3.357200] mmc_spi spi0.0: SD/MMC host mmc0, no DMA, no WP, no poweroff, cd polling
[    3.367366] usbcore: registered new interface driver usbhid
[    3.372272] usbhid: USB HID core driver
[    3.379571] NET: Registered protocol family 17
[    3.389575] Loading compiled-in X.509 certificates
[    3.507181] mmc0: host does not support reading read-only switch, assuming write-enable
[    3.514622] mmc0: new SDHC card on SPI
[    3.522097] mmcblk0: mmc0:0000 SL08G 7.40 GiB 
[    3.610215] Freeing unused kernel memory: 11908K
[    3.614166] This architecture does not have kernel memory protection.
Starting logging: OK
Starting mdev...
sort: /sys/devices/platform/Fixed: No such file or directory
modprobe: can't change directory to '/lib/modules': No such file or directory
Initializing random number generator... done.
Starting network...
Waiting for interface eth0 to appear............... timeout!
run-parts: /etc/network/if-pre-up.d/wait_iface: exit status 1
Starting dropbear sshd: OK

Welcome to Buildroot
buildroot login: root
Password: 
# ls
aes.riscv               fiveroot.sh             sidroot.sh
attestation.eapp_riscv  fiveroot.sh~            sidroot.sh~
demo.sh                 keystone-driver.ko      stack.eapp_riscv
enclave-host.riscv      long-nop.eapp_riscv     test
eyrie-rt                loop.eapp_riscv         test-runner.riscv
fib-bench.eapp_riscv    malloc.eapp_riscv       trusted_client.riscv
fibonacci.eapp_riscv    server_eapp.eapp_riscv  untrusted.eapp_riscv
# insmod keystone-driver.ko 
[   43.904785] keystone_driver: loading out-of-tree module taints kernel.
[   43.920765] keystone_enclave: keystone enclave v0.2
# ./test
testing stack
testing fibonacci
[   96.446790] INFO: rcu_sched detected stalls on CPUs/tasks:
[   96.451741]  1-...0: (6 ticks this GP) idle=cc6/140000000000000/0 softirq=2157/2157 fqs=1051 
[   96.460158]  (detected by 0, t=2103 jiffies, g=95, c=94, q=0)
[   96.465902] Task dump for CPU 1:
[   96.469120] test-runner.ris R  running task        0   126    124 0x00000008
[   96.476146] Call Trace:
[   96.478699] [<        (ptrval)>] ret_from_fork+0x0/0xc
testing long-nop
testing loop
[  152.706790] INFO: rcu_sched detected stalls on CPUs/tasks:
[  152.711743]  0-...0: (6 ticks this GP) idle=292/140000000000000/0 softirq=1760/1760 fqs=1051 
[  152.720158]  (detected by 1, t=2103 jiffies, g=106, c=105, q=0)
[  152.726078] Task dump for CPU 0:
[  152.729292] test-runner.ris R  running task        0   128    124 0x00000008
[  152.736321] Call Trace:
[  152.738863] [<        (ptrval)>] ret_from_fork+0x0/0xc
testing malloc
[  176.556786] INFO: rcu_sched detected stalls on CPUs/tasks:
[  176.561738]  0-...0: (1 GPs behind) idle=3f6/140000000000000/0 softirq=1773/1774 fqs=1051 
[  176.569899]  (detected by 3, t=2103 jiffies, g=112, c=111, q=1)
[  176.575811] Task dump for CPU 0:
[  176.579029] test-runner.ris R  running task        0   129    124 0x00000008
[  176.586057] Call Trace:
[  176.588608] [<        (ptrval)>] ret_from_fork+0x0/0xc
testing fib-bench
[  202.287145] INFO: rcu_sched self-detected stall on CPU
[  202.291748]  2-....: (2099 ticks this GP) idle=82e/140000000000001/0 softirq=869/869 fqs=11 
[  202.300081]   (t=2100 jiffies g=116 c=115 q=2)
[  202.304531] Task dump for CPU 2:
[  202.307742] test-runner.ris R  running task        0   130    124 0x00000008
[  202.314772] Call Trace:
[  202.317336] [<        (ptrval)>] walk_stackframe+0x0/0xa2
[  202.322662] [<        (ptrval)>] show_stack+0x26/0x34
[  202.327722] [<        (ptrval)>] sched_show_task+0xa8/0x108
[  202.333258] [<        (ptrval)>] dump_cpu_task+0x3a/0x48
[  202.338557] [<        (ptrval)>] rcu_dump_cpu_stacks+0x7a/0xb8
[  202.344373] [<        (ptrval)>] rcu_check_callbacks+0x52c/0x644
[  202.350384] [<        (ptrval)>] update_process_times+0x1e/0x4c
[  202.356296] [<        (ptrval)>] tick_periodic+0x40/0xac
[  202.361583] [<        (ptrval)>] tick_handle_periodic+0x56/0x5c
[  202.367518] [<        (ptrval)>] riscv_timer_interrupt+0x26/0x32
[  202.373510] [<        (ptrval)>] riscv_intc_irq+0x58/0xee
[  202.378890] [<        (ptrval)>] ret_from_syscall+0xa/0xe
[  202.384398] [<        (ptrval)>] keystone_finalize_enclave+0x14/0xe4 [keystone_driver]
testing untrusted
[  233.056786] INFO: rcu_sched detected stalls on CPUs/tasks:
[  233.061744]  0-...0: (6 ticks this GP) idle=b4e/140000000000000/0 softirq=1795/1795 fqs=1051 
[  233.070157]  (detected by 3, t=2103 jiffies, g=122, c=121, q=1)
[  233.076076] Task dump for CPU 0:
[  233.079294] test-runner.ris R  running task        0   131    124 0x00000008
[  233.086320] Call Trace:
[  233.088867] [<        (ptrval)>] ret_from_fork+0x0/0xc
Enclave said: hello world!
Enclave said: 2nd hello world!
Enclave said value: 13
Enclave said value: 20
testing attestation
[  258.913162] INFO: rcu_sched self-detected stall on CPU
[  258.917772]  2-....: (2099 ticks this GP) idle=2a6/140000000000001/0 softirq=887/887 fqs=12 
[  258.926102]   (t=2100 jiffies g=126 c=125 q=2)
[  258.930550] Task dump for CPU 2:
[  258.933764] test-runner.ris R  running task        0   132    124 0x00000008
[  258.940794] Call Trace:
[  258.943348] [<        (ptrval)>] walk_stackframe+0x0/0xa2
[  258.948687] [<        (ptrval)>] show_stack+0x26/0x34
[  258.953743] [<        (ptrval)>] sched_show_task+0xa8/0x108
[  258.959270] [<        (ptrval)>] dump_cpu_task+0x3a/0x48
[  258.964575] [<        (ptrval)>] rcu_dump_cpu_stacks+0x7a/0xb8
[  258.970392] [<        (ptrval)>] rcu_check_callbacks+0x52c/0x644
[  258.976406] [<        (ptrval)>] update_process_times+0x1e/0x4c
[  258.982318] [<        (ptrval)>] tick_periodic+0x40/0xac
[  258.987606] [<        (ptrval)>] tick_handle_periodic+0x56/0x5c
[  258.993536] [<        (ptrval)>] riscv_timer_interrupt+0x26/0x32
[  258.999541] [<        (ptrval)>] riscv_intc_irq+0x58/0xee
[  259.004912] [<        (ptrval)>] ret_from_syscall+0xa/0xe
[  259.010418] [<        (ptrval)>] keystone_finalize_enclave+0x14/0xe4 [keystone_driver]
Attestation report SIGNATURE is valid
# 

[tmagik]

Have a look at https://github.com/tmagik/freedom/blob/bootrom_hacks/bootrom/uboot/Makefile

[mcd500]

Your link is going to support different dts for every different boards for the kernel. e.g. Unleashed, vc707, vcu118. It is great. When the vc707 or Unleashed have extention boards or peripherals added, then probably using device tree overlay is the genuine dts way. which we could work on it some time in the future. Thanks for the link.