Debugging applications with litex_sim

g2gps commented 1 year ago

Hello,

I'm attempting to use litex_sim to debug an application running on VexRiscv. I've been using the wiki pages: Use-GDB-with-VexRiscv-CPU, Use-Host-Bridge-to-control-debug-a-SoC and SoC-Simulator as a guide, however I might not fully understand the implementation or limitations. Here's my workflow:

Run the simulator, with my application.

$ litex_sim --integrated-main-ram-size=0x10000000 --cpu-type=vexriscv --ram-init=boot.json --cpu-variant=standard+debug --csr-csv=csr.csv --with-ethernet --with-etherbone --sim-debug --ethernet-phy-model=sim

Which, after building, gives the following:

[gmii_ethernet] loaded (0x561335fbfef0)
[spdeeprom] loaded (addr = 0x0)
[clocker] loaded
[serial2tcp] loaded (0x561335fbfef0)
[serial2console] loaded (0x561335fbfef0)
[ethernet] loaded (0x561335fbfef0)
[xgmii_ethernet] loaded (0x561335fbfef0)
[clocker] sys_clk: freq_hz=1000000, phase_deg=0

        __   _ __      _  __
       / /  (_) /____ | |/_/
      / /__/ / __/ -_)>  <
     /____/_/\__/\__/_/|_|
   Build your hardware, easily!

 (c) Copyright 2012-2022 Enjoy-Digital
 (c) Copyright 2007-2015 M-Labs

 BIOS built on Mar  1 2023 12:07:29
 BIOS CRC passed (0ad00533)

 LiteX git sha1: dcb54b85

--=============== SoC ==================--
CPU:            VexRiscv_Debug @ 1MHz
BUS:            WISHBONE 32-bit @ 4GiB
CSR:            32-bit data
ROM:            128.0KiB
SRAM:           8.0KiB
MAIN-RAM:       256.0MiB

--========== Initialization ============--
Ethernet init...

--============== Boot ==================--
Booting from serial...
Press Q or ESC to abort boot completely.
sL5DdSMmkekro
Timeout
Executing booted program at 0x40000000

--============= Liftoff! ===============--

NuttShell (NSH) NuttX-12.0.0
nsh>

That looks like it's working as expected. I can also ping the exposed IP address:

$ ping 192.168.1.100
PING 192.168.1.100 (192.168.1.100) 56(84) bytes of data.
64 bytes from 192.168.1.100: icmp_seq=1 ttl=64 time=0.016 ms
64 bytes from 192.168.1.100: icmp_seq=2 ttl=64 time=0.029 ms

I made a slight modification to litex_sim.py, as I was getting UnboundLocalError: local variable 'ethmac' referenced before assignment

--- a/litex/tools/litex_sim.py
+++ b/litex/tools/litex_sim.py
@@ -226,9 +226,9 @@ class SimSoC(SoCCore):
                 hw_mac     = etherbone_mac_address)

             # SoftCPU
-            ethmac_region_size = (ethmac.rx_slots.constant + ethmac.tx_slots.constant)*ethmac.slot_size.constant
+            ethmac_region_size = (self.ethmac.rx_slots.constant + self.ethmac.tx_slots.constant)*self.ethmac.slot_size.constant
             ethmac_region = SoCRegion(origin=self.mem_map.get("ethmac", None), size=ethmac_region_size, cached=False)
-            self.bus.add_slave(name="ethmac", slave=ethmac.bus, region=ethmac_region)
+            self.bus.add_slave(name="ethmac", slave=self.ethmac.bus, region=ethmac_region)
             if self.irq.enabled:
                 self.irq.add("ethmac", use_loc_if_exists=True)
             # HW ethernet
diff --git a/litex/tools/remote/comm_udp.py b/litex/tools/remote/comm_udp.py
index e282a660..0ce4379c 100644
--- a/litex/tools/remote/comm_udp.py
+++ b/litex/tools/remote/comm_udp.py

After that, it falls over when I run litex_server:

$ litex_server --udp --udp-ip=192.168.1.100
[CommUDP] ip: 192.168.1.100 / port: 1234 / tcp port: 1234

Traceback (most recent call last):
  File "/home/xx/.local/bin/litex_server", line 33, in <module>
    sys.exit(load_entry_point('litex', 'console_scripts', 'litex_server')())
  File "/home/xx/projects/xx/litex/litex/tools/litex_server.py", line 283, in main
    server.open()
  File "/home/xx/projects/xx/litex/litex/tools/litex_server.py", line 90, in open
    self.comm.open()
  File "/home/xx/projects/xx/litex/litex/tools/remote/comm_udp.py", line 34, in open
    self.probe(self.server, self.port)
  File "/home/xx/projects/xx/litex/litex/tools/remote/comm_udp.py", line 73, in probe
    assert packet.pr == 1
AssertionError

Possibly I'm missing some configuration to get this up and running? Let me know if any further info is required.

g2gps commented 1 year ago

Just to elaborate. Our end goal is to be able to debug our application using litex_sim and the vexriscv_smp soft core. Is that something which is currently feasible?

I've noticed that there are no cpu-variant flags with "+debug" for the veriscv_smp core. Is that hinting at there not being any support for it currently?

Dolu1990 commented 1 year ago

Which kind of debug you want to setup ? GDB based, while the MMU is used ? (to understandthe whole picture)

enjoy-digital commented 1 year ago

Hi @g2gps,

thanks for reporting the compilation issue, this is fixed with: https://github.com/enjoy-digital/litex/commit/ea2171d32bb784a1c9749ca8683b90e71a3f2427.

In this simulation, the IP addresses are:

192.168.1.50: Ethernet MAC.
192.168.1.51: Etherbone.

So for the debug, you'll want to connect to the Etherbone: litex_server --udp --udp-ip=192.168.1.51

The debug process should then be similar to the one described in Use-GDB-with-VexRiscv-CPU. But this will only work for VexRiscv, for VexRiscv-SMP this will be different and not sure we have tested it through one of the LiteX bridges.

g2gps commented 1 year ago

Thanks @enjoy-digital. litex_server now connects initially, but fails once openocd attempts to connect. My workflow:

litex_sim

litex_sim --integrated-main-ram-size=0x10000000 --cpu-type=vexriscv --cpu-variant=standard+debug --ram-init=nuttx.bin  --csr-csv=csr.csv --with-ethernet --with-etherbone --sim-debug

# Output as in above example

litex_server

$ litex_server --udp --udp-ip=192.168.1.51 --debug
[CommUDP] ip: 192.168.1.51 / port: 1234 / tcp port: 1234

# Output after openocd attempts to connect

Connected with 127.0.0.1:58334
read 0x4000d584 @ 0xf00f0000
write 0x12300013 @ 0xf00f0004
read 0x40804dd8 @ 0xf00f0004
Disconnect
Exception in thread Thread-1 (_serve_thread):
Traceback (most recent call last):
  File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.10/threading.py", line 953, in run
    self._target(*self._args, **self._kwargs)
  File "/home/xx/projects/xx/litex/litex/tools/litex_server.py", line 162, in _serve_thread
    self.send_packet(client_socket, packet)
  File "/home/xx/projects/xx/litex/litex/tools/remote/etherbone.py", line 363, in send_packet
    socket.sendall(packet.bytes)
BrokenPipeError: [Errno 32] Broken pipe

openocd

./src/openocd -c 'interface dummy' \
              -c 'adapter_khz 1' \
              -c 'jtag newtap lx cpu -irlen 4' \
              -c 'target create lx.cpu0 vexriscv -endian little -chain-position lx.cpu -dbgbase 0xF00F0000' \
              -c 'vexriscv cpuConfigFile cpu0.yaml' \
              -c 'vexriscv networkProtocol etherbone' \
              -c 'init' \
              -c 'reset halt'
Open On-Chip Debugger 0.11.0+dev-02577-g3eee6eb04 (2023-03-02-09:32)
Licensed under GNU GPL v2
For bug reports, read
        http://openocd.org/doc/doxygen/bugs.html
DEPRECATED! use 'adapter driver' not 'interface'
Info : only one transport option; autoselect 'jtag'
adapter speed: 1 kHz

4027514880
Info : clock speed 1 kHz
Info : TAP lx.cpu does not have valid IDCODE (idcode=0x0)
Info : TAP auto0.tap does not have valid IDCODE (idcode=0x80000000)
Info : TAP auto1.tap does not have valid IDCODE (idcode=0xc0000000)
Info : TAP auto2.tap does not have valid IDCODE (idcode=0xe0000000)
Info : TAP auto3.tap does not have valid IDCODE (idcode=0xf0000000)
Info : TAP auto4.tap does not have valid IDCODE (idcode=0xf8000000)
Info : TAP auto5.tap does not have valid IDCODE (idcode=0xfc000000)
Info : TAP auto6.tap does not have valid IDCODE (idcode=0xfe000000)
Info : TAP auto7.tap does not have valid IDCODE (idcode=0xff000000)
Info : TAP auto8.tap does not have valid IDCODE (idcode=0xff800000)
Info : TAP auto9.tap does not have valid IDCODE (idcode=0xffc00000)
Info : TAP auto10.tap does not have valid IDCODE (idcode=0xffe00000)
Info : TAP auto11.tap does not have valid IDCODE (idcode=0xfff00000)
Info : TAP auto12.tap does not have valid IDCODE (idcode=0xfff80000)
Info : TAP auto13.tap does not have valid IDCODE (idcode=0xfffc0000)
Info : TAP auto14.tap does not have valid IDCODE (idcode=0xfffe0000)
Info : TAP auto15.tap does not have valid IDCODE (idcode=0xffff0000)
Info : TAP auto16.tap does not have valid IDCODE (idcode=0xffff8000)
Info : TAP auto17.tap does not have valid IDCODE (idcode=0xffffc000)
Info : TAP auto18.tap does not have valid IDCODE (idcode=0xffffe000)
Info : TAP auto19.tap does not have valid IDCODE (idcode=0xfffff000)
Warn : Unexpected idcode after end of chain: 21 0xfffff800
Error: double-check your JTAG setup (interface, speed, ...)
Error: Trying to use configured scan chain anyway...
Error: lx.cpu: IR capture error; saw 0x0f not 0x01
Warn : Bypassing JTAG setup events due to errors
Error: remote bridge closed network connection

A couple of things I've investigated:

I can see the vesriscv_debug region in my csr.csv: memory_region,vexriscv_debug,0xf00f0000,256,io
The same workflow, instead using the litex_bare_metal_demo. Same result.
I've tried openocd with the version outlined in "Use-GDB-with-VexRiscv-CPU" and the latest available.

I'm not sure if it something specific to my setup. Happy to provide further info if needed.

g2gps commented 1 year ago

@Dolu1990 Yes. The overall goal would to be able to use gdb to debug, using the VexRiscv-SMP core, with MMU enabled.

enjoy-digital commented 1 year ago

The work done by @gatecat in https://github.com/enjoy-digital/litex/pull/1636 could also be useful for this. @gatecat if you have the possibility to share what you ended with, I could have a look to integrate.

OrkunAliOzkan commented 1 year ago

Hello,

I'm attempting to use litex_sim to debug an application running on VexRiscv ... Here's my workflow:

Run the simulator, with my application.
$ litex_sim --integrated-main-ram-size=0x10000000 --cpu-type=vexriscv --ram-init=boot.json --cpu-variant=standard+debug --csr-csv=csr.csv --with-ethernet --with-etherbone --sim-debug --ethernet-phy-model=sim

Hello, I am currently trying to figure out how to preload all dependencies needed for booting linux onto a simulated SoC (Image, fw_jump.bin, initrd_bb for example). How exactly do the contents mapped out on boot.json get uploaded in litex_sim?

(If asking underneath is rude etiquette do let me know I am not too acquainted with customs on open source projects but am more than happy to learn them!)

AndrewD commented 1 year ago

I just got sim debug working with etherbone: it works with wishbone-tool but litex_server seems to have a regression which prevents the combination of litex_server and openocd from working. wishbone-tool is a simpler setup in the end too.

enjoy-digital / litex

Debugging applications with litex_sim #1626