search

litex-hub / linux-on-litex-vexriscv

Linux on LiteX-VexRiscv
BSD 2-Clause "Simplified" License
574 stars 174 forks source link

KCU105 Build Error: [Place 30-1114] Floating OSERDES ODDRE1_1 is not placed #238

Closed garrettxw closed 3 years ago

garrettxw commented 3 years ago

Hello! I am trying to build the bitstream for the KCU105 board using ./make.py --board=kcu105 --cpu-count=4 --build, however, it fails in the place design step like so:

19 Infos, 1 Warnings, 0 Critical Warnings and 10 Errors encountered.
place_design completed successfully
place_design: Time (s): cpu = 00:07:28 ; elapsed = 00:03:50 . Memory (MB): peak = 3580.902 ; gain = 16.008 ; free physical = 3456 ; free virtual = 8132
ERROR: [Common 17-39] 'place_design' failed due to earlier errors.

    while executing
"place_design -directive default"
    (file "kcu105.tcl" line 46)
INFO: [Common 17-206] Exiting Vivado at Wed Jul  7 14:15:02 2021...
Traceback (most recent call last):
  File "./make.py", line 662, in <module>
    main()
  File "./make.py", line 640, in main
    builder.build(run=args.build, build_name=board_name)
  File "/home/garrett/Litex/litex/litex/soc/integration/builder.py", line 293, in build
    vns = self.soc.build(build_dir=self.gateware_dir, **kwargs)
  File "/home/garrett/Litex/litex/litex/soc/integration/soc.py", line 1113, in build
    return self.platform.build(self, *args, **kwargs)
  File "/home/garrett/Litex/litex/litex/build/xilinx/platform.py", line 53, in build
    return self.toolchain.build(self, *args, **kwargs)
  File "/home/garrett/Litex/litex/litex/build/xilinx/vivado.py", line 355, in build
    _run_script(script)
  File "/home/garrett/Litex/litex/litex/build/xilinx/vivado.py", line 101, in _run_script
    raise OSError("Error occured during Vivado's script execution.")
OSError: Error occured during Vivado's script execution.

I reran it in the Vivado GUI using the commands from the kcu105.tcl file and it gave me 10 errors after failing the place design step again: 

[Place 30-1114] Floating OSERDES ODDRE1_1 is not placed and is not a part of any Shape. It won't be placed by the IO Placer.

Repeating with OSERDES ODDRE1_(2-9) After looking up similar issues, I suspect it might have something to do with my Vivado version or perhaps my Ubuntu version. I have used both Vivado 2020.2 and 2018.1 on Ubuntu 20.04.2 LTS 64-bit.

Edit: I have also tried it on Vivado 2019.1 and Ununtu 18.04

garrettxw commented 3 years ago

For clarification, it would be useful to know what version of Vivado the build scripts are meant to be ran with for the KCU105 board

garrettxw commented 3 years ago

I just tried running it in Vivado 2019.1 and it failed with the same error, however I was able to catch where it happened more specifically during the placing task: 

Phase 1.2 IO Placement/ Clock Placement/ Build Placer Device
INFO: [Timing 38-35] Done setting XDC timing constraints.

Followed by 10 errors that look the same as the one in my original post.

I also tried building the Nexys Video bitstream on a bit of a whim, since we're using those boards in our lab, and it built correctly.

joshuadenis commented 3 years ago

This is a bit late, but I've run into the same problem. After a lot of digging around, and piecing together different resources (Don't have all the links anymore, sorry), what worked for me was modifying the script that generates the build instructions for Vivado.

In the file litex/litex/build/xilinx/vivado.py at line 230 in the placement section right before the line tcl.append("place_design -directive {}".format(self.vivado_place_directive)), I added tcl.append("place_ports"). Everything now builds correctly.

I am running Vivado 2018.1 on Ubuntu 21.04.

Hope this helps!

garrettxw commented 3 years ago

Just tried this and it works! Thanks a ton. My coworker looked at it and ignoring the DRC check error by adding set_property SEVERITY {Warning} [get_drc_checks REQP-1753] in the pre-placement commands part of the kcu105.tcl script also seems to work. He also had me disable the SD card on line 174 of make.py since it wasn't placed properly which is something that should be addressed.

garrettxw commented 3 years ago

Ok I've got it built, but now I'm getting an error when trying to run the Linux image. Not sure if its related to the previous error or if its another thing with the KCU105

litex> reboot

        __   _ __      _  __
       / /  (_) /____ | |/_/
      / /__/ / __/ -_)>  <
     /____/_/\__/\__/_/|_|
   Build your hardware, easily!

 (c) Copyright 2012-2021 Enjoy-Digital
 (c) Copyright 2007-2015 M-Labs

 BIOS built on Jul 19 2021 13:39:16
 BIOS CRC passed (5d3ea6ae)

 Migen git sha1: 3ffd64c
 LiteX git sha1: dd5413bc

--=============== SoC ==================--
CPU:            VexRiscv SMP-LINUX @ 125MHz
BUS:            WISHBONE 32-bit @ 4GiB
CSR:            32-bit data
ROM:            64KiB
SRAM:           8KiB
L2:             0KiB
SDRAM:          1048576KiB 64-bit @ 1000MT/s (CL-9 CWL-9)

--========== Initialization ============--
Ethernet init...
Initializing SDRAM @0x40000000...
Switching SDRAM to software control.
Write leveling:
  tCK equivalent taps: 436
  Cmd/Clk scan (0-218)
  |1111  |11111  |11111  |11111| best: 0
  Setting Cmd/Clk delay to 0 taps.
  Data scan:
  m0: |00000000111111111111110000| delay: 115
  m1: |00000000011111111111111000| delay: 130
  m2: |00000000000111111111111110| delay: 162
  m3: |00000000000011111111111110| delay: 179
  m4: |00000000000011111111111111| delay: 187
  m5: |00000000000001111111111111| delay: 195
  m6: |00000000000001111111111111| delay: 205
  m7: |10000000000000111111111111| delay: 215
Write latency calibration:
m0:0 m1:0 m2:0 m3:0 m4:0 m5:0 m6:0 m7:0 
Read leveling:
  m0, b00: |00000000000000000000000000000000| delays: -
  m0, b01: |00000000000000000000000000000000| delays: -
  m0, b02: |01111111111110000000000000000000| delays: 103+-95
  m0, b03: |00000000000000111111111111000000| delays: 320+-95
  m0, b04: |00000000000000000000000000000111| delays: 481+-30
  m0, b05: |00000000000000000000000000000000| delays: -
  m0, b06: |00000000000000000000000000000000| delays: -
  m0, b07: |00000000000000000000000000000000| delays: -
  best: m0, b02 delays: 99+-98
  m1, b00: |00000000000000000000000000000000| delays: -
  m1, b01: |00000000000000000000000000000000| delays: -
  m1, b02: |01111111111110000000000000000000| delays: 108+-96
  m1, b03: |00000000000000011111111111100000| delays: 326+-94
  m1, b04: |00000000000000000000000000000111| delays: 480+-31
  m1, b05: |00000000000000000000000000000000| delays: -
  m1, b06: |00000000000000000000000000000000| delays: -
  m1, b07: |00000000000000000000000000000000| delays: -
  best: m1, b02 delays: 108+-97
  m2, b00: |00000000000000000000000000000000| delays: -
  m2, b01: |00000000000000000000000000000000| delays: -
  m2, b02: |11111111111000000000000000000000| delays: 85+-85
  m2, b03: |00000000000001111111111110000000| delays: 297+-95
  m2, b04: |00000000000000000000000000011111| delays: 468+-43
  m2, b05: |00000000000000000000000000000000| delays: -
  m2, b06: |00000000000000000000000000000000| delays: -
  m2, b07: |00000000000000000000000000000000| delays: -
  best: m2, b03 delays: 299+-95
  m3, b00: |00000000000000000000000000000000| delays: -
  m3, b01: |00000000000000000000000000000000| delays: -
  m3, b02: |01111111111100000000000000000000| delays: 100+-93
  m3, b03: |00000000000000111111111111000000| delays: 316+-91
  m3, b04: |00000000000000000000000000001111| delays: 478+-34
  m3, b05: |00000000000000000000000000000000| delays: -
  m3, b06: |00000000000000000000000000000000| delays: -
  m3, b07: |00000000000000000000000000000000| delays: -
  best: m3, b03 delays: 318+-96
  m4, b00: |00000000000000000000000000000000| delays: -
  m4, b01: |00000000000000000000000000000000| delays: -
  m4, b02: |11111111110000000000000000000000| delays: 81+-81
  m4, b03: |00000000000001111111111100000000| delays: 286+-91
  m4, b04: |00000000000000000000000000011111| delays: 464+-48
  m4, b05: |00000000000000000000000000000000| delays: -
  m4, b06: |00000000000000000000000000000000| delays: -
  m4, b07: |00000000000000000000000000000000| delays: -
  best: m4, b03 delays: 286+-90
  m5, b00: |00000000000000000000000000000000| delays: -
  m5, b01: |00000000000000000000000000000000| delays: -
  m5, b02: |11111111110000000000000000000000| delays: 76+-76
  m5, b03: |00000000000011111111111100000000| delays: 280+-91
  m5, b04: |00000000000000000000000000011111| delays: 464+-48
  m5, b05: |00000000000000000000000000000000| delays: -
  m5, b06: |00000000000000000000000000000000| delays: -
  m5, b07: |00000000000000000000000000000000| delays: -
  best: m5, b03 delays: 282+-93
  m6, b00: |00000000000000000000000000000000| delays: -
  m6, b01: |00000000000000000000000000000000| delays: -
  m6, b02: |11111111000000000000000000000000| delays: 61+-61
  m6, b03: |00000000001111111111100000000000| delays: 245+-90
  m6, b04: |00000000000000000000000011111111| delays: 447+-65
  m6, b05: |00000000000000000000000000000000| delays: -
  m6, b06: |00000000000000000000000000000000| delays: -
  m6, b07: |00000000000000000000000000000000| delays: -
  best: m6, b03 delays: 247+-90
  m7, b00: |00000000000000000000000000000000| delays: -
  m7, b01: |00000000000000000000000000000000| delays: -
  m7, b02: |11111111100000000000000000000000| delays: 65+-65
  m7, b03: |00000000000111111111110000000000| delays: 252+-92
  m7, b04: |00000000000000000000000001111111| delays: 448+-64
  m7, b05: |00000000000000000000000000000000| delays: -
  m7, b06: |00000000000000000000000000000000| delays: -
  m7, b07: |00000000000000000000000000000000| delays: -
  best: m7, b03 delays: 250+-90
Switching SDRAM to hardware control.
Memtest at 0x40000000 (2.0MiB)...
  Write: 0x40000000-0x40200000 2.0MiB     
   Read: 0x40000000-0x40200000 2.0MiB     
  bus errors:  256/256
  addr errors: 8174/8192
  data errors: 524288/524288
Memtest KO
Memory initialization failed

--============= Console ================--

Looking at it, the thing that stands out to me is the SDRAM size, it says it has 1048576 KiB (1 GiB) while the KCU105 specs have 2 GB of DDR4

garrettxw commented 3 years ago

So, I got it working but I'm not exactly sure how. I was messing around with old commits trying to see if something between the first commit that the KCU105 was working on and the current commit broke it. While doing so I checked to see if the newest commit was still not working on a whim and it worked! I'll try to narrow down what could have done it and post it here later.

AEW2015 commented 3 years ago

I figured this out. The main issue is that for SDR I/O it is trying to use the ODDRE1 primitive that translates to an OSERDES3 prrimitive. According to/nmigen/nmigen/vendor/xilinx_ultrascale.py it should use

            # SDR I/O is performed by packing a flip-flop into the pad IOB.
            for bit in range(len(q)):
                m.submodules += Instance("FDCE",
                    a_IOB="TRUE",
                    i_C=clk,
                    i_CE=Const(1),
                    i_CLR=Const(0),
                    i_D=d[bit],
                    o_Q=q[bit]
                )

Here is my git diff to litex repo to get it to work

diff --git a/litex/build/xilinx/common.py b/litex/build/xilinx/common.py
index c4ee2e40..3b43431d 100644
--- a/litex/build/xilinx/common.py
+++ b/litex/build/xilinx/common.py
@@ -343,18 +343,39 @@ class XilinxDDRInputUS:

 # Ultrascale SDROutput -----------------------------------------------------------------------------

+class XilinxSDROutputImplUS(Module):
+    def __init__(self, i, o, clk):
+        self.specials += Instance("FDCE",
+            i_C=clk,
+            i_CE=1,
+            i_CLR=0,
+            i_D=i,
+            o_Q=o
+        )
+
 class XilinxSDROutputUS:
     @staticmethod
     def lower(dr):
-        return XilinxDDROutputImplUS(dr.i, dr.i, dr.o, dr.clk)
+        return XilinxSDROutputImplUS(dr.i, dr.o, dr.clk)
+        

 # Ultrascale SDRInput ------------------------------------------------------------------------------
+class XilinxSDRInputImplUS(Module):
+    def __init__(self, i, o, clk): 
+        self.specials += Instance("FDCE",
+            i_C=clk,
+            i_CE=1,
+            i_CLR=0,
+            i_D=i,
+            o_Q=o
+        )
+

 class XilinxSDRInputUS:
     @staticmethod
     def lower(dr):
-        return XilinxDDRInputImplUS(dr.i, dr.o, Signal(), dr.clk)
+        return XilinxSDRInputImplUS(dr.i, dr.o, dr.clk)

 # Ultrascale Specials Overrides --------------------------------------------------------------------

I tried to apply the IOB feature in Verilog and the XDC, but it either didn't work or caused a DRC failure.

benoit1995 commented 3 years ago

I figured this out. The main issue is that for SDR I/O it is trying to use the ODDRE1 primitive that translates to an OSERDES3 prrimitive. According to/nmigen/nmigen/vendor/xilinx_ultrascale.py it should use

            # SDR I/O is performed by packing a flip-flop into the pad IOB.
            for bit in range(len(q)):
                m.submodules += Instance("FDCE",
                    a_IOB="TRUE",
                    i_C=clk,
                    i_CE=Const(1),
                    i_CLR=Const(0),
                    i_D=d[bit],
                    o_Q=q[bit]
                )

Here is my git diff to litex repo to get it to work

diff --git a/litex/build/xilinx/common.py b/litex/build/xilinx/common.py
index c4ee2e40..3b43431d 100644
--- a/litex/build/xilinx/common.py
+++ b/litex/build/xilinx/common.py
@@ -343,18 +343,39 @@ class XilinxDDRInputUS:

 # Ultrascale SDROutput -----------------------------------------------------------------------------

+class XilinxSDROutputImplUS(Module):
+    def __init__(self, i, o, clk):
+        self.specials += Instance("FDCE",
+            i_C=clk,
+            i_CE=1,
+            i_CLR=0,
+            i_D=i,
+            o_Q=o
+        )
+
 class XilinxSDROutputUS:
     @staticmethod
     def lower(dr):
-        return XilinxDDROutputImplUS(dr.i, dr.i, dr.o, dr.clk)
+        return XilinxSDROutputImplUS(dr.i, dr.o, dr.clk)
+        

 # Ultrascale SDRInput ------------------------------------------------------------------------------
+class XilinxSDRInputImplUS(Module):
+    def __init__(self, i, o, clk): 
+        self.specials += Instance("FDCE",
+            i_C=clk,
+            i_CE=1,
+            i_CLR=0,
+            i_D=i,
+            o_Q=o
+        )
+

 class XilinxSDRInputUS:
     @staticmethod
     def lower(dr):
-        return XilinxDDRInputImplUS(dr.i, dr.o, Signal(), dr.clk)
+        return XilinxSDRInputImplUS(dr.i, dr.o, dr.clk)

 # Ultrascale Specials Overrides --------------------------------------------------------------------

I tried to apply the IOB feature in Verilog and the XDC, but it either didn't work or caused a DRC failure.

Hi, Have you successfully used the KCU105 sdcard boot kernel? I followed your method successfully, but I can't boot the kernel with sdcard.

AEW2015 commented 3 years ago

Here is my fork with this: https://github.com/AEW2015/litex/commit/2059e6530716fa2e3828f7a6dc37a54ddd77dc43 I got it working with the AES-KU040 board with a pmod SDCARD. We figured out the KCU105 SDcard is controlled by the ZYNQ microcontroller and not readable from the KU040 without messing with the boot modes. The only way we got it to boot was to put it in flash boot mode with the bit file in the internal flash, and then it can read the sdcard. (The mode pins have to be set before power-up).

benoit1995 commented 3 years ago

Here is my fork with this: AEW2015/litex@2059e65 I got it working with the AES-KU040 board with a pmod SDCARD. We figured out the KCU105 SDcard is controlled by the ZYNQ microcontroller and not readable from the KU040 without messing with the boot modes. The only way we got it to boot was to put it in flash boot mode with the bit file in the internal flash, and then it can read the sdcard. (The mode pins have to be set before power-up).

Hi, Thank you for your reply.

I have successfully booted linux using SDCard. According to the following link, SYSCTLR_SDIO_MUX_SEL is set to 1 via the zynq interface (by selecting Configure UltraScale FPGA, and then pressing 0 to return) . https://forums.xilinx.com/t5/Xilinx-Evaluation-Boards/KCU105-micro-SD-slot-SDIO-mux/m-p/759972

After downloading the bitfile we can successfully access the SDCard.

Thank you again for the code

AEW2015 commented 3 years ago

So I would get a mutli-meter and prob the SDcard enable line that goes to the MUX for the scard from the ZYNQ system controller. Try all the variations of boot modes and figure out when that signal changes.

enjoy-digital commented 3 years ago

Thanks @AEW2015 for the fix, this has been integrated in LiteX: https://github.com/enjoy-digital/litex/commit/9f75c73d6ba9a86fd4479b7f8666ae619bf1d1a3. The issue was introduced by the use of SDRTristate in LiteSDCard.