search

enjoy-digital / litex

Build your hardware, easily!
Other
2.99k stars 568 forks source link

Cannot start Litex_term on/dev/ttyLXU0 (PCIe) when running Linux on Acorn #1550

Open PatrickChenHZ opened 1 year ago

PatrickChenHZ commented 1 year ago

Hi, I have been stuck on this issue for days where I built the linux for acorn board but cannot connect to bios or upload linux image via litex_term. More specifically my host computer would just lock up/freeze when I try to use litex_term on the virtual UART port created by litex_pcie driver. The driver loaded fine, and litexpcie_util can correctly see FPGA info and do DMA test. I didn't observer any major error when building the bitstream. Sorry I am failing to provide more specific info on where the issue is, because I just straight up don't know. It could be FPGA side or PCIe driver side or litex_term, but there is no error message showed when lock up happens, and it happens every single time. I attached syslog when lockup happens below, hope that will help, since that's all I got.

Board:Acorn_CLE215 Project:linux-on-litex-vexriscv Build command used: ./make.py --board=acorn_pcie --build --flash System lock up when executing: litex_term /dev/ttyLXU0 or litex_term --image wheremyimageis /dev/ttyLXU0 Host operating system ubuntu 22.04 (also tried on different computer with ubuntu 18.04 same result) Host computer: Intel NUC PCIe Connection: Thunderbolt3 (also tried directly attach to motherboard with same result)

syslog when computer crashed:

Jan  3 02:18:50 patrickchen-NUC10i7FNH rtkit-daemon[934]: Supervising 7 threads of 4 processes of 1 users.
Jan  3 02:19:50 patrickchen-NUC10i7FNH rtkit-daemon[934]: message repeated 3 times: [ Supervising 7 threads of 4 processes of 1 users.]
Jan  3 02:20:10 patrickchen-NUC10i7FNH kernel: [ 2011.842655] litepcie: loading out-of-tree module taints kernel.
Jan  3 02:20:10 patrickchen-NUC10i7FNH kernel: [ 2011.842687] litepcie: module verification failed: signature and/or required key missing - tainting kernel
Jan  3 02:20:10 patrickchen-NUC10i7FNH kernel: [ 2011.843039] litepcie 0000:06:00.0: [Probing device]
Jan  3 02:20:10 patrickchen-NUC10i7FNH kernel: [ 2011.843054] litepcie 0000:06:00.0: enabling device (0000 -> 0002)
Jan  3 02:20:10 patrickchen-NUC10i7FNH kernel: [ 2011.863504] litepcie 0000:06:00.0: Version 
Jan  3 02:20:10 patrickchen-NUC10i7FNH kernel: [ 2011.863583] litepcie 0000:06:00.0: 1 MSI IRQs allocated.
Jan  3 02:20:10 patrickchen-NUC10i7FNH kernel: [ 2011.863595] litepcie 0000:06:00.0: Creating /dev/litepcie0
Jan  3 02:20:10 patrickchen-NUC10i7FNH kernel: [ 2011.866648] liteuart.1: ttyLXU0 at MMIO 0x0 (irq = 0, base_baud = 0) is a liteuart
Jan  3 02:20:28 patrickchen-NUC10i7FNH gnome-shell[1697]: libinput error: event10 - MX Keys Keyboard: client bug: event processing lagging behind by 29ms, your system is too slow
Jan  3 02:20:46 patrickchen-NUC10i7FNH nautilus[4678]: Called "net usershare info" but it failed: Failed to execute child process “net” (No such file or directory)
Jan  3 02:20:52 patrickchen-NUC10i7FNH rtkit-daemon[934]: Supervising 7 threads of 4 processes of 1 users.
Jan  3 02:20:52 patrickchen-NUC10i7FNH rtkit-daemon[934]: Supervising 7 threads of 4 processes of 1 users.
Jan  3 02:21:32 patrickchen-NUC10i7FNH gnome-shell[1697]: Window manager warning: last_user_time (2093887) is greater than comparison timestamp (2092697).  This most likely represents a buggy client sending inaccurate timestamps in messages such as _NET_ACTIVE_WINDOW.  Trying to work around...
Jan  3 02:21:32 patrickchen-NUC10i7FNH gnome-shell[1697]: Window manager warning: W20 appears to be one of the offending windows with a timestamp of 2093887.  Working around...
Jan  3 02:21:35 patrickchen-NUC10i7FNH gnome-shell[1697]: Window manager warning: last_user_time (2094449) is greater than comparison timestamp (2094179).  This most likely represents a buggy client sending inaccurate timestamps in messages such as _NET_ACTIVE_WINDOW.  Trying to work around...
Jan  3 02:21:35 patrickchen-NUC10i7FNH gnome-shell[1697]: Window manager warning: W20 appears to be one of the offending windows with a timestamp of 2094449.  Working around...
Jan  3 02:21:38 patrickchen-NUC10i7FNH kernel: [ 2100.237551] watchdog: BUG: soft lockup - CPU#10 stuck for 26s! [swapper/10:0]
Jan  3 02:21:38 patrickchen-NUC10i7FNH kernel: [ 2100.237557] Modules linked in: liteuart(OE) litepcie(OE) ftdi_sio pcspkr ch341 usbserial rfcomm joydev input_leds hid_generic uhid hid cmac algif_hash algif_skcipher af_alg snd_hda_codec_hdmi snd_sof_pci_intel_cnl snd_sof_intel_hda_common soundwire_intel soundwire_generic_allocation soundwire_cadence snd_sof_intel_hda snd_sof_pci snd_sof_xtensa_dsp snd_sof snd_soc_hdac_hda snd_hda_ext_core snd_ctl_led snd_soc_acpi_intel_match snd_hda_codec_realtek snd_soc_acpi soundwire_bus snd_hda_codec_generic bnep ledtrig_audio snd_soc_core snd_compress ac97_bus snd_pcm_dmaengine intel_rapl_msr intel_rapl_common snd_hda_intel x86_pkg_temp_thermal snd_intel_dspcfg snd_intel_sdw_acpi snd_hda_codec intel_powerclamp snd_hda_core snd_hwdep coretemp snd_pcm kvm_intel snd_seq_midi binfmt_misc iwlmvm snd_seq_midi_event kvm tps6598x i915 mei_hdcp typec mac80211 snd_rawmidi crct10dif_pclmul ghash_clmulni_intel libarc4 btusb ttm btrtl snd_seq btbcm aesni_intel drm_kms_helper nls_iso8859_1 crypto_simd btintel
Jan  3 02:21:38 patrickchen-NUC10i7FNH kernel: [ 2100.237592]  cryptd snd_seq_device cec iwlwifi bluetooth snd_timer rc_core rapl i2c_algo_bit fb_sys_fops intel_cstate intel_wmi_thunderbolt wmi_bmof snd ecdh_generic syscopyarea cfg80211 ee1004 ecc soundcore sysfillrect sysimgblt mei_me intel_pch_thermal mei serial_multi_instantiate mac_hid acpi_tad acpi_pad sch_fq_codel msr parport_pc ppdev lp drm parport ramoops pstore_blk reed_solomon pstore_zone efi_pstore ip_tables x_tables autofs4 sdhci_pci cqhci nvme i2c_i801 crc32_pclmul e1000e thunderbolt sdhci nvme_core i2c_smbus intel_lpss_pci xhci_pci ahci intel_lpss libahci idma64 xhci_pci_renesas wmi video pinctrl_cannonlake
Jan  3 02:21:38 patrickchen-NUC10i7FNH kernel: [ 2100.237620] CPU: 10 PID: 0 Comm: swapper/10 Tainted: G           OE     5.15.0-56-generic #62-Ubuntu
Jan  3 02:21:38 patrickchen-NUC10i7FNH kernel: [ 2100.237622] Hardware name: Intel(R) Client Systems NUC10i7FNH/NUC10i7FNB, BIOS FNCML357.0055.2021.1202.1748 12/02/2021
Jan  3 02:21:38 patrickchen-NUC10i7FNH kernel: [ 2100.237623] RIP: 0010:liteuart_timer+0x8d/0x100 [liteuart]
Jan  3 02:21:38 patrickchen-NUC10i7FNH kernel: [ 2100.237628] Code: ff 00 00 00 00 45 31 c0 44 89 e9 31 d2 be 01 00 00 00 4c 89 ff e8 c3 08 b0 c5 49 8b bc 24 f0 fe ff ff e8 36 46 ae c5 41 8b 06 <84> c0 74 9d 41 8b 84 24 40 ff ff ff 83 f8 06 7f 40 b8 01 00 00 00
Jan  3 02:21:38 patrickchen-NUC10i7FNH kernel: [ 2100.237629] RSP: 0018:ffffb65ac037ce80 EFLAGS: 00000202
Jan  3 02:21:38 patrickchen-NUC10i7FNH kernel: [ 2100.237631] RAX: 0000000000000000 RBX: ffffb65ac3d06820 RCX: ffff8bc41cfe3a10
Jan  3 02:21:38 patrickchen-NUC10i7FNH kernel: [ 2100.237632] RDX: 0000000000000000 RSI: 0000000000000006 RDI: ffff8bc400051000
Jan  3 02:21:38 patrickchen-NUC10i7FNH kernel: [ 2100.237633] RBP: ffffb65ac037cea8 R08: 0000000000000001 R09: ffffffff886898d8
Jan  3 02:21:38 patrickchen-NUC10i7FNH kernel: [ 2100.237633] R10: 0000000000000003 R11: 000000000000000a R12: ffff8bc417318200
Jan  3 02:21:38 patrickchen-NUC10i7FNH kernel: [ 2100.237634] R13: 0000000000000000 R14: ffffb65ac3d06828 R15: ffff8bc417318028
Jan  3 02:21:38 patrickchen-NUC10i7FNH kernel: [ 2100.237635] FS:  0000000000000000(0000) GS:ffff8bc7b0e80000(0000) knlGS:0000000000000000
Jan  3 02:21:38 patrickchen-NUC10i7FNH kernel: [ 2100.237636] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jan  3 02:21:38 patrickchen-NUC10i7FNH kernel: [ 2100.237637] CR2: 00007f5eb5c06658 CR3: 0000000268e10003 CR4: 00000000003706e0
Jan  3 02:21:38 patrickchen-NUC10i7FNH kernel: [ 2100.237638] Call Trace:
Jan  3 02:21:38 patrickchen-NUC10i7FNH kernel: [ 2100.237639]  <IRQ>
Jan  3 02:21:38 patrickchen-NUC10i7FNH kernel: [ 2100.237641]  ? liteuart_startup+0x70/0x70 [liteuart]
Jan  3 02:21:38 patrickchen-NUC10i7FNH kernel: [ 2100.237643]  call_timer_fn+0x29/0x120
Jan  3 02:21:38 patrickchen-NUC10i7FNH kernel: [ 2100.237646]  __run_timers.part.0+0x1e3/0x270
Jan  3 02:21:38 patrickchen-NUC10i7FNH kernel: [ 2100.237648]  ? ktime_get+0x43/0xc0
Jan  3 02:21:38 patrickchen-NUC10i7FNH kernel: [ 2100.237650]  ? lapic_next_deadline+0x2c/0x40
Jan  3 02:21:38 patrickchen-NUC10i7FNH kernel: [ 2100.237652]  ? clockevents_program_event+0xaa/0x130
Jan  3 02:21:38 patrickchen-NUC10i7FNH kernel: [ 2100.237655]  run_timer_softirq+0x2a/0x60
Jan  3 02:21:38 patrickchen-NUC10i7FNH kernel: [ 2100.237656]  __do_softirq+0xd6/0x2e7
Jan  3 02:21:38 patrickchen-NUC10i7FNH kernel: [ 2100.237659]  irq_exit_rcu+0x94/0xc0
Jan  3 02:21:38 patrickchen-NUC10i7FNH kernel: [ 2100.237662]  sysvec_apic_timer_interrupt+0x80/0x90
Jan  3 02:21:38 patrickchen-NUC10i7FNH kernel: [ 2100.237664]  </IRQ>
Jan  3 02:21:38 patrickchen-NUC10i7FNH kernel: [ 2100.237665]  <TASK>
Jan  3 02:21:38 patrickchen-NUC10i7FNH kernel: [ 2100.237665]  asm_sysvec_apic_timer_interrupt+0x1b/0x20
Jan  3 02:21:38 patrickchen-NUC10i7FNH kernel: [ 2100.237666] RIP: 0010:cpuidle_enter_state+0xd9/0x620
Jan  3 02:21:38 patrickchen-NUC10i7FNH kernel: [ 2100.237670] Code: 3d 44 d8 d9 78 e8 d7 c2 68 ff 49 89 c7 0f 1f 44 00 00 31 ff e8 18 d0 68 ff 80 7d d0 00 0f 85 61 01 00 00 fb 66 0f 1f 44 00 00 <45> 85 f6 0f 88 6d 01 00 00 4d 63 ee 49 83 fd 09 0f 87 e7 03 00 00
Jan  3 02:21:38 patrickchen-NUC10i7FNH kernel: [ 2100.237671] RSP: 0018:ffffb65ac0163e28 EFLAGS: 00000246
Jan  3 02:21:38 patrickchen-NUC10i7FNH kernel: [ 2100.237672] RAX: ffff8bc7b0eb0b80 RBX: ffffd65abfc80f00 RCX: 0000000000000000
Jan  3 02:21:38 patrickchen-NUC10i7FNH kernel: [ 2100.237672] RDX: 0000000000000000 RSI: 0000000000000002 RDI: 0000000000000000
Jan  3 02:21:38 patrickchen-NUC10i7FNH kernel: [ 2100.237673] RBP: ffffb65ac0163e78 R08: 000001e3185d0ca6 R09: 00000000002f5530
Jan  3 02:21:38 patrickchen-NUC10i7FNH kernel: [ 2100.237674] R10: 0000000000000004 R11: 071c71c71c71c71c R12: ffffffff888d40e0
Jan  3 02:21:38 patrickchen-NUC10i7FNH kernel: [ 2100.237675] R13: 0000000000000002 R14: 0000000000000002 R15: 000001e3185d0ca6
Jan  3 02:21:38 patrickchen-NUC10i7FNH kernel: [ 2100.237677]  ? cpuidle_enter_state+0xc8/0x620
Jan  3 02:21:38 patrickchen-NUC10i7FNH kernel: [ 2100.237679]  cpuidle_enter+0x2e/0x50
Jan  3 02:21:38 patrickchen-NUC10i7FNH kernel: [ 2100.237681]  cpuidle_idle_call+0x142/0x1e0
Jan  3 02:21:38 patrickchen-NUC10i7FNH kernel: [ 2100.237683]  do_idle+0x83/0xf0
Jan  3 02:21:38 patrickchen-NUC10i7FNH kernel: [ 2100.237684]  cpu_startup_entry+0x20/0x30
Jan  3 02:21:38 patrickchen-NUC10i7FNH kernel: [ 2100.237685]  start_secondary+0x12a/0x180
Jan  3 02:21:38 patrickchen-NUC10i7FNH kernel: [ 2100.237687]  secondary_startup_64_no_verify+0xc2/0xcb
Jan  3 02:21:38 patrickchen-NUC10i7FNH kernel: [ 2100.237689]  </TASK>
Jan  3 02:21:52 patrickchen-NUC10i7FNH rtkit-daemon[934]: Supervising 7 threads of 4 processes of 1 users.
Jan  3 02:21:52 patrickchen-NUC10i7FNH rtkit-daemon[934]: Supervising 7 threads of 4 processes of 1 users.
Jan  3 02:21:56 patrickchen-NUC10i7FNH gnome-shell[1697]: Window manager warning: Ping serial 2118249 was reused for window W9, previous use was for window W0.
Jan  3 02:22:06 patrickchen-NUC10i7FNH kernel: [ 2128.237242] watchdog: BUG: soft lockup - CPU#10 stuck for 53s! [swapper/10:0]
Jan  3 02:22:06 patrickchen-NUC10i7FNH kernel: [ 2128.237247] Modules linked in: liteuart(OE) litepcie(OE) ftdi_sio pcspkr ch341 usbserial rfcomm joydev input_leds hid_generic uhid hid cmac algif_hash algif_skcipher af_alg snd_hda_codec_hdmi snd_sof_pci_intel_cnl snd_sof_intel_hda_common soundwire_intel soundwire_generic_allocation soundwire_cadence snd_sof_intel_hda snd_sof_pci snd_sof_xtensa_dsp snd_sof snd_soc_hdac_hda snd_hda_ext_core snd_ctl_led snd_soc_acpi_intel_match snd_hda_codec_realtek snd_soc_acpi soundwire_bus snd_hda_codec_generic bnep ledtrig_audio snd_soc_core snd_compress ac97_bus snd_pcm_dmaengine intel_rapl_msr intel_rapl_common snd_hda_intel x86_pkg_temp_thermal snd_intel_dspcfg snd_intel_sdw_acpi snd_hda_codec intel_powerclamp snd_hda_core snd_hwdep coretemp snd_pcm kvm_intel snd_seq_midi binfmt_misc iwlmvm snd_seq_midi_event kvm tps6598x i915 mei_hdcp typec mac80211 snd_rawmidi crct10dif_pclmul ghash_clmulni_intel libarc4 btusb ttm btrtl snd_seq btbcm aesni_intel drm_kms_helper nls_iso8859_1 crypto_simd btintel
Jan  3 02:22:06 patrickchen-NUC10i7FNH kernel: [ 2128.237320]  cryptd snd_seq_device cec iwlwifi bluetooth snd_timer rc_core rapl i2c_algo_bit fb_sys_fops intel_cstate intel_wmi_thunderbolt wmi_bmof snd ecdh_generic syscopyarea cfg80211 ee1004 ecc soundcore sysfillrect sysimgblt mei_me intel_pch_thermal mei serial_multi_instantiate mac_hid acpi_tad acpi_pad sch_fq_codel msr parport_pc ppdev lp drm parport ramoops pstore_blk reed_solomon pstore_zone efi_pstore ip_tables x_tables autofs4 sdhci_pci cqhci nvme i2c_i801 crc32_pclmul e1000e thunderbolt sdhci nvme_core i2c_smbus intel_lpss_pci xhci_pci ahci intel_lpss libahci idma64 xhci_pci_renesas wmi video pinctrl_cannonlake
Jan  3 02:22:06 patrickchen-NUC10i7FNH kernel: [ 2128.237347] CPU: 10 PID: 0 Comm: swapper/10 Tainted: G           OEL    5.15.0-56-generic #62-Ubuntu
Jan  3 02:22:06 patrickchen-NUC10i7FNH kernel: [ 2128.237366] Hardware name: Intel(R) Client Systems NUC10i7FNH/NUC10i7FNB, BIOS FNCML357.0055.2021.1202.1748 12/02/2021
Jan  3 02:22:06 patrickchen-NUC10i7FNH kernel: [ 2128.237367] RIP: 0010:liteuart_timer+0x8d/0x100 [liteuart]
Jan  3 02:22:06 patrickchen-NUC10i7FNH kernel: [ 2128.237371] Code: ff 00 00 00 00 45 31 c0 44 89 e9 31 d2 be 01 00 00 00 4c 89 ff e8 c3 08 b0 c5 49 8b bc 24 f0 fe ff ff e8 36 46 ae c5 41 8b 06 <84> c0 74 9d 41 8b 84 24 40 ff ff ff 83 f8 06 7f 40 b8 01 00 00 00
Jan  3 02:22:06 patrickchen-NUC10i7FNH kernel: [ 2128.237372] RSP: 0018:ffffb65ac037ce80 EFLAGS: 00000202
Jan  3 02:22:06 patrickchen-NUC10i7FNH kernel: [ 2128.237373] RAX: 0000000000000000 RBX: ffffb65ac3d06820 RCX: 0000000000030b80
Jan  3 02:22:06 patrickchen-NUC10i7FNH kernel: [ 2128.237374] RDX: 0000000000000000 RSI: 0000000000000006 RDI: ffff8bc400051000
Jan  3 02:22:06 patrickchen-NUC10i7FNH kernel: [ 2128.237375] RBP: ffffb65ac037cea8 R08: 0000000000000001 R09: 0000000000000008
Jan  3 02:22:06 patrickchen-NUC10i7FNH kernel: [ 2128.237376] R10: 0000000000000002 R11: 000000000000000a R12: ffff8bc417318200
Jan  3 02:22:06 patrickchen-NUC10i7FNH kernel: [ 2128.237376] R13: 0000000000000000 R14: ffffb65ac3d06828 R15: ffff8bc417318028
Jan  3 02:22:06 patrickchen-NUC10i7FNH kernel: [ 2128.237377] FS:  0000000000000000(0000) GS:ffff8bc7b0e80000(0000) knlGS:0000000000000000
Jan  3 02:22:06 patrickchen-NUC10i7FNH kernel: [ 2128.237378] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jan  3 02:22:06 patrickchen-NUC10i7FNH kernel: [ 2128.237379] CR2: 00007f5eb5c06658 CR3: 0000000268e10003 CR4: 00000000003706e0
Jan  3 02:22:06 patrickchen-NUC10i7FNH kernel: [ 2128.237380] Call Trace:
Jan  3 02:22:06 patrickchen-NUC10i7FNH kernel: [ 2128.237381]  <IRQ>
Jan  3 02:22:06 patrickchen-NUC10i7FNH kernel: [ 2128.237382]  ? liteuart_startup+0x70/0x70 [liteuart]
Jan  3 02:22:06 patrickchen-NUC10i7FNH kernel: [ 2128.237384]  call_timer_fn+0x29/0x120
Jan  3 02:22:06 patrickchen-NUC10i7FNH kernel: [ 2128.237387]  __run_timers.part.0+0x1e3/0x270
Jan  3 02:22:06 patrickchen-NUC10i7FNH kernel: [ 2128.237388]  ? ktime_get+0x43/0xc0
Jan  3 02:22:06 patrickchen-NUC10i7FNH kernel: [ 2128.237390]  ? lapic_next_deadline+0x2c/0x40
Jan  3 02:22:06 patrickchen-NUC10i7FNH kernel: [ 2128.237392]  ? clockevents_program_event+0xaa/0x130
Jan  3 02:22:06 patrickchen-NUC10i7FNH kernel: [ 2128.237394]  run_timer_softirq+0x2a/0x60
Jan  3 02:22:06 patrickchen-NUC10i7FNH kernel: [ 2128.237395]  __do_softirq+0xd6/0x2e7
Jan  3 02:22:06 patrickchen-NUC10i7FNH kernel: [ 2128.237397]  irq_exit_rcu+0x94/0xc0
Jan  3 02:22:06 patrickchen-NUC10i7FNH kernel: [ 2128.237400]  sysvec_apic_timer_interrupt+0x80/0x90
Jan  3 02:22:06 patrickchen-NUC10i7FNH kernel: [ 2128.237402]  </IRQ>
Jan  3 02:22:06 patrickchen-NUC10i7FNH kernel: [ 2128.237402]  <TASK>
Jan  3 02:22:06 patrickchen-NUC10i7FNH kernel: [ 2128.237403]  asm_sysvec_apic_timer_interrupt+0x1b/0x20
Jan  3 02:22:06 patrickchen-NUC10i7FNH kernel: [ 2128.237404] RIP: 0010:cpuidle_enter_state+0xd9/0x620
Jan  3 02:22:06 patrickchen-NUC10i7FNH kernel: [ 2128.237407] Code: 3d 44 d8 d9 78 e8 d7 c2 68 ff 49 89 c7 0f 1f 44 00 00 31 ff e8 18 d0 68 ff 80 7d d0 00 0f 85 61 01 00 00 fb 66 0f 1f 44 00 00 <45> 85 f6 0f 88 6d 01 00 00 4d 63 ee 49 83 fd 09 0f 87 e7 03 00 00
Jan  3 02:22:06 patrickchen-NUC10i7FNH kernel: [ 2128.237407] RSP: 0018:ffffb65ac0163e28 EFLAGS: 00000246
Jan  3 02:22:06 patrickchen-NUC10i7FNH kernel: [ 2128.237408] RAX: ffff8bc7b0eb0b80 RBX: ffffd65abfc80f00 RCX: 0000000000000000
Jan  3 02:22:06 patrickchen-NUC10i7FNH kernel: [ 2128.237409] RDX: 0000000000000000 RSI: 0000000000000002 RDI: 0000000000000000
Jan  3 02:22:06 patrickchen-NUC10i7FNH kernel: [ 2128.237410] RBP: ffffb65ac0163e78 R08: 000001e3185d0ca6 R09: 00000000002f5530
Jan  3 02:22:06 patrickchen-NUC10i7FNH kernel: [ 2128.237410] R10: 0000000000000004 R11: 071c71c71c71c71c R12: ffffffff888d40e0
Jan  3 02:22:06 patrickchen-NUC10i7FNH kernel: [ 2128.237411] R13: 0000000000000002 R14: 0000000000000002 R15: 000001e3185d0ca6
Jan  3 02:22:06 patrickchen-NUC10i7FNH kernel: [ 2128.237413]  ? cpuidle_enter_state+0xc8/0x620
Jan  3 02:22:06 patrickchen-NUC10i7FNH kernel: [ 2128.237414]  cpuidle_enter+0x2e/0x50
Jan  3 02:22:06 patrickchen-NUC10i7FNH kernel: [ 2128.237416]  cpuidle_idle_call+0x142/0x1e0
Jan  3 02:22:06 patrickchen-NUC10i7FNH kernel: [ 2128.237417]  do_idle+0x83/0xf0
Jan  3 02:22:06 patrickchen-NUC10i7FNH kernel: [ 2128.237419]  cpu_startup_entry+0x20/0x30
Jan  3 02:22:06 patrickchen-NUC10i7FNH kernel: [ 2128.237420]  start_secondary+0x12a/0x180
Jan  3 02:22:06 patrickchen-NUC10i7FNH kernel: [ 2128.237421]  secondary_startup_64_no_verify+0xc2/0xcb
Jan  3 02:22:06 patrickchen-NUC10i7FNH kernel: [ 2128.237423]  </TASK>
Jan  3 02:22:13 patrickchen-NUC10i7FNH kernel: [ 2134.877170] rcu: INFO: rcu_sched self-detected stall on CPU
Jan  3 02:22:13 patrickchen-NUC10i7FNH kernel: [ 2134.877174] rcu:  10-....: (1 GPs behind) idle=2ff/1/0x4000000000000004 softirq=65947/65952 fqs=7037 
Jan  3 02:22:13 patrickchen-NUC10i7FNH kernel: [ 2134.877176]   (t=15000 jiffies g=142785 q=9974)
Jan  3 02:22:13 patrickchen-NUC10i7FNH kernel: [ 2134.877178] NMI backtrace for cpu 10
Jan  3 02:22:13 patrickchen-NUC10i7FNH kernel: [ 2134.877179] CPU: 10 PID: 0 Comm: swapper/10 Tainted: G           OEL    5.15.0-56-generic #62-Ubuntu
Jan  3 02:22:13 patrickchen-NUC10i7FNH kernel: [ 2134.877181] Hardware name: Intel(R) Client Systems NUC10i7FNH/NUC10i7FNB, BIOS FNCML357.0055.2021.1202.1748 12/02/2021
Jan  3 02:22:13 patrickchen-NUC10i7FNH kernel: [ 2134.877181] Call Trace:
Jan  3 02:22:13 patrickchen-NUC10i7FNH kernel: [ 2134.877210]  <IRQ>
Jan  3 02:22:13 patrickchen-NUC10i7FNH kernel: [ 2134.877212]  show_stack+0x52/0x5c
Jan  3 02:22:13 patrickchen-NUC10i7FNH kernel: [ 2134.877215]  dump_stack_lvl+0x4a/0x63
Jan  3 02:22:13 patrickchen-NUC10i7FNH kernel: [ 2134.877219]  dump_stack+0x10/0x16
Jan  3 02:22:13 patrickchen-NUC10i7FNH kernel: [ 2134.877221]  nmi_cpu_backtrace.cold+0x4d/0x93
Jan  3 02:22:13 patrickchen-NUC10i7FNH kernel: [ 2134.877223]  ? lapic_can_unplug_cpu+0x90/0x90
Jan  3 02:22:13 patrickchen-NUC10i7FNH kernel: [ 2134.877226]  nmi_trigger_cpumask_backtrace+0xec/0x100
Jan  3 02:22:13 patrickchen-NUC10i7FNH kernel: [ 2134.877229]  arch_trigger_cpumask_backtrace+0x19/0x20
Jan  3 02:22:13 patrickchen-NUC10i7FNH kernel: [ 2134.877231]  trigger_single_cpu_backtrace+0x44/0x4f
Jan  3 02:22:13 patrickchen-NUC10i7FNH kernel: [ 2134.877234]  rcu_dump_cpu_stacks+0x102/0x149
Jan  3 02:22:13 patrickchen-NUC10i7FNH kernel: [ 2134.877235]  print_cpu_stall.cold+0x2f/0xe2
Jan  3 02:22:13 patrickchen-NUC10i7FNH kernel: [ 2134.877237]  ? timekeeping_update+0xf7/0x150
Jan  3 02:22:13 patrickchen-NUC10i7FNH kernel: [ 2134.877240]  check_cpu_stall+0x109/0x270
Jan  3 02:22:13 patrickchen-NUC10i7FNH kernel: [ 2134.877242]  rcu_sched_clock_irq+0x9a/0x250
Jan  3 02:22:13 patrickchen-NUC10i7FNH kernel: [ 2134.877261]  update_process_times+0x94/0xd0
Jan  3 02:22:13 patrickchen-NUC10i7FNH kernel: [ 2134.877262]  ? tick_nohz_handler+0xb0/0xb0
Jan  3 02:22:13 patrickchen-NUC10i7FNH kernel: [ 2134.877264]  tick_sched_handle+0x29/0x70
Jan  3 02:22:13 patrickchen-NUC10i7FNH kernel: [ 2134.877265]  ? tick_nohz_handler+0xb0/0xb0
Jan  3 02:22:13 patrickchen-NUC10i7FNH kernel: [ 2134.877266]  tick_sched_timer+0x8c/0xc0
Jan  3 02:22:13 patrickchen-NUC10i7FNH kernel: [ 2134.877267]  __hrtimer_run_queues+0x106/0x210
Jan  3 02:22:13 patrickchen-NUC10i7FNH kernel: [ 2134.877268]  hrtimer_interrupt+0x101/0x220
Jan  3 02:22:13 patrickchen-NUC10i7FNH kernel: [ 2134.877287]  __sysvec_apic_timer_interrupt+0x5e/0xe0
Jan  3 02:22:13 patrickchen-NUC10i7FNH kernel: [ 2134.877288]  sysvec_apic_timer_interrupt+0x3e/0x90
Jan  3 02:22:13 patrickchen-NUC10i7FNH kernel: [ 2134.877290]  asm_sysvec_apic_timer_interrupt+0x1b/0x20
Jan  3 02:22:13 patrickchen-NUC10i7FNH kernel: [ 2134.877292] RIP: 0010:liteuart_timer+0x8d/0x100 [liteuart]
Jan  3 02:22:13 patrickchen-NUC10i7FNH kernel: [ 2134.877296] Code: ff 00 00 00 00 45 31 c0 44 89 e9 31 d2 be 01 00 00 00 4c 89 ff e8 c3 08 b0 c5 49 8b bc 24 f0 fe ff ff e8 36 46 ae c5 41 8b 06 <84> c0 74 9d 41 8b 84 24 40 ff ff ff 83 f8 06 7f 40 b8 01 00 00 00
Jan  3 02:22:13 patrickchen-NUC10i7FNH kernel: [ 2134.877297] RSP: 0018:ffffb65ac037ce80 EFLAGS: 00000202
Jan  3 02:22:13 patrickchen-NUC10i7FNH kernel: [ 2134.877298] RAX: 0000000000000000 RBX: ffffb65ac3d06820 RCX: 0000000000030b80
Jan  3 02:22:13 patrickchen-NUC10i7FNH kernel: [ 2134.877299] RDX: 0000000000000000 RSI: 0000000000000006 RDI: ffff8bc400051000
Jan  3 02:22:13 patrickchen-NUC10i7FNH kernel: [ 2134.877300] RBP: ffffb65ac037cea8 R08: 0000000000000001 R09: 0000000000000008
Jan  3 02:22:13 patrickchen-NUC10i7FNH kernel: [ 2134.877301] R10: 0000000000000006 R11: 000000000000000a R12: ffff8bc417318200
Jan  3 02:22:13 patrickchen-NUC10i7FNH kernel: [ 2134.877301] R13: 0000000000000000 R14: ffffb65ac3d06828 R15: ffff8bc417318028
Jan  3 02:22:13 patrickchen-NUC10i7FNH kernel: [ 2134.877303]  ? liteuart_startup+0x70/0x70 [liteuart]
Jan  3 02:22:13 patrickchen-NUC10i7FNH kernel: [ 2134.877304]  call_timer_fn+0x29/0x120
Jan  3 02:22:13 patrickchen-NUC10i7FNH kernel: [ 2134.877305]  __run_timers.part.0+0x1e3/0x270
Jan  3 02:22:13 patrickchen-NUC10i7FNH kernel: [ 2134.877306]  ? ktime_get+0x43/0xc0
Jan  3 02:22:13 patrickchen-NUC10i7FNH kernel: [ 2134.877308]  ? lapic_next_deadline+0x2c/0x40
Jan  3 02:22:13 patrickchen-NUC10i7FNH kernel: [ 2134.877309]  ? clockevents_program_event+0xaa/0x130
Jan  3 02:22:13 patrickchen-NUC10i7FNH kernel: [ 2134.877311]  run_timer_softirq+0x2a/0x60
Jan  3 02:22:13 patrickchen-NUC10i7FNH kernel: [ 2134.877312]  __do_softirq+0xd6/0x2e7
Jan  3 02:22:13 patrickchen-NUC10i7FNH kernel: [ 2134.877314]  irq_exit_rcu+0x94/0xc0
Jan  3 02:22:13 patrickchen-NUC10i7FNH kernel: [ 2134.877316]  sysvec_apic_timer_interrupt+0x80/0x90
Jan  3 02:22:13 patrickchen-NUC10i7FNH kernel: [ 2134.877318]  </IRQ>
Jan  3 02:22:13 patrickchen-NUC10i7FNH kernel: [ 2134.877318]  <TASK>
Jan  3 02:22:13 patrickchen-NUC10i7FNH kernel: [ 2134.877319]  asm_sysvec_apic_timer_interrupt+0x1b/0x20
Jan  3 02:22:13 patrickchen-NUC10i7FNH kernel: [ 2134.877320] RIP: 0010:cpuidle_enter_state+0xd9/0x620
Jan  3 02:22:13 patrickchen-NUC10i7FNH kernel: [ 2134.877322] Code: 3d 44 d8 d9 78 e8 d7 c2 68 ff 49 89 c7 0f 1f 44 00 00 31 ff e8 18 d0 68 ff 80 7d d0 00 0f 85 61 01 00 00 fb 66 0f 1f 44 00 00 <45> 85 f6 0f 88 6d 01 00 00 4d 63 ee 49 83 fd 09 0f 87 e7 03 00 00
Jan  3 02:22:13 patrickchen-NUC10i7FNH kernel: [ 2134.877323] RSP: 0018:ffffb65ac0163e28 EFLAGS: 00000246
Jan  3 02:22:13 patrickchen-NUC10i7FNH kernel: [ 2134.877324] RAX: ffff8bc7b0eb0b80 RBX: ffffd65abfc80f00 RCX: 0000000000000000
Jan  3 02:22:13 patrickchen-NUC10i7FNH kernel: [ 2134.877325] RDX: 0000000000000000 RSI: 0000000000000002 RDI: 0000000000000000
Jan  3 02:22:13 patrickchen-NUC10i7FNH kernel: [ 2134.877325] RBP: ffffb65ac0163e78 R08: 000001e3185d0ca6 R09: 00000000002f5530
Jan  3 02:22:13 patrickchen-NUC10i7FNH kernel: [ 2134.877326] R10: 0000000000000004 R11: 071c71c71c71c71c R12: ffffffff888d40e0
Jan  3 02:22:13 patrickchen-NUC10i7FNH kernel: [ 2134.877327] R13: 0000000000000002 R14: 0000000000000002 R15: 000001e3185d0ca6
Jan  3 02:22:13 patrickchen-NUC10i7FNH kernel: [ 2134.877328]  ? cpuidle_enter_state+0xc8/0x620
Jan  3 02:22:13 patrickchen-NUC10i7FNH kernel: [ 2134.877329]  cpuidle_enter+0x2e/0x50
Jan  3 02:22:13 patrickchen-NUC10i7FNH kernel: [ 2134.877331]  cpuidle_idle_call+0x142/0x1e0
Jan  3 02:22:13 patrickchen-NUC10i7FNH kernel: [ 2134.877332]  do_idle+0x83/0xf0
Jan  3 02:22:13 patrickchen-NUC10i7FNH kernel: [ 2134.877334]  cpu_startup_entry+0x20/0x30
Jan  3 02:22:13 patrickchen-NUC10i7FNH kernel: [ 2134.877335]  start_secondary+0x12a/0x180
Jan  3 02:22:13 patrickchen-NUC10i7FNH kernel: [ 2134.877336]  secondary_startup_64_no_verify+0xc2/0xcb
Jan  3 02:22:13 patrickchen-NUC10i7FNH kernel: [ 2134.877338]  </TASK>

Thank you!

enjoy-digital commented 1 year ago

Hi @PatrickChenHZ,

I can probably help on this and have the hardware at hand. Can you send attach your bistream + litepcie kernel (including csr.h). I'll do a test to see if it's related to the hardware or to the bistream/software header generation.

I could also share a version working on my setup to do tests on yours.

PatrickChenHZ commented 1 year ago

kernel.zip gateware.zip

Thank you so much!

sthornington commented 6 months ago

Just wanted to add that I am in the same spot. However, I can add that creating the target directly from litex-boards, I can get litex_term to see the BIOS works fine. I’m going to check whether adding more flags to the AcornPCIE target, when launched from make.py, to correspond to how I ran the script on the command line (e.g. —driver), produces working drivers.

sthornington commented 6 months ago

In what may or may not be a coincidence, I’m also using a NUC 10 i7FNH with the Acorn connected via a Thunderbolt bay for the M.2 card.

sthornington commented 6 months ago

Did you ever figure this out @PatrickChenHZ ?

sthornington commented 6 months ago

Something definitely weird with the Linux-on-litex build, I wonder if it’s something to do with the operational/fallback stuff. In that documented anywhere?

Raw build from the target in litex-boards seems to work reliably, but once flashed with the Linux-on-litex version, the PCIe stuff is sketchy, sometimes the board shuts down completely, with the fan turning off and the led chaser stopping even.

sthornington commented 6 months ago

Okay I got it to load in "safe" mode, via litex_term --images=images/boot.json --safe /dev/ttyLXU0. I also, personally, changed the make.py to not specifically use the fallback bitstream when flashing, since I think that might also have been causing problems?