search

roboticslab-uc3m / asibot-main

ASIBOT assistive robot: super/meta repository.
http://roboticslab.uc3m.es/roboticslab/robot/asibot
GNU Lesser General Public License v2.1
4 stars 0 forks source link

Peak-CAN comms fail with IRQ error #44

Closed PeterBowman closed 2 years ago

PeterBowman commented 5 years ago

ASIBOT has undergone a major SO upgrade from old Debian 6.0 ("squeeze") to Debian 8.0 ("jessie"), see https://github.com/roboticslab-uc3m/questions-and-answers/issues/20. Everything went fine but the ability to read/write via CAN. Perhaps this is linked to the fact that I installed old HiCO.CAN kernel drivers first, checked that they worked, and then tried the Peak CAN ones. It seems that initial communication succeeds within a CAN-based application (e.g. oneCanBusOneWrapper), but it is not capable of performing CAN transfers afterwards.

I traced this error down to kernel log messages and found an obscure "irq 9: nobody cared (try booting with the "irqpoll" option)". I reached out to PEAK support and obtained no answer. I'm pasting here the same kernel log excerpts shared with support, plus this SO link: https://stackoverflow.com/q/13861282.

Hello. I'm reaching out to you regarding a strange kernel message that is shown up right after I start my CAN application:

kernel: [ 185.154899] Disabling IRQ #9

The app succeeds in establishing a communication (receives and transmits CAN frames) only for a few seconds, then it freezes. Same error is observed upon launching PCAN's test apps.

dmesg output:

[ 154.449561] irq 9: nobody cared (try booting with the "irqpoll" option)
[ 154.449652] CPU: 0 PID: 718 Comm: oneCanBusOneWra Tainted: G W O 3.16.0-6-586 #1 Debian 3.16.57-2
[ 154.449672] ce407f80 ce407f80 c107f899 c1525a80 00000009 00000009 ce404480 00000000
[ 154.449716] ce407fa4 c107fb94 fbb27658 00000000 ce404480 00000000 ce404480 00000009
[ 154.449753] 00000000 ce407fe0 c107deb2 c1080020 ce407fd0 c144ecc7 00000000 00000007
[ 154.449791] Call Trace:
[ 154.449839] [<c107f899>] ? __report_bad_irq.isra.6+0x29/0xc0
[ 154.449866] [<c107fb94>] ? note_interrupt+0x204/0x240
[ 154.449907] [<c107deb2>] ? handle_irq_event_percpu+0x132/0x1a0
[ 154.449932] [<c1080020>] ? handle_edge_irq+0x100/0x100
[ 154.449970] [<c144ecc7>] ? printk+0x1c/0x21
[ 154.449996] [<c1080020>] ? handle_edge_irq+0x100/0x100
[ 154.450023] [<c107df41>] ? handle_irq_event+0x21/0x30
[ 154.450047] [<c108007f>] ? handle_level_irq+0x5f/0xe0
[ 154.450075] [<c1003f63>] ? handle_irq+0xa3/0xc0
[ 154.450088] <IRQ> [<c1453df9>] ? do_IRQ+0x39/0xc0
[ 154.450166] [<d0810f9b>] ? r6040_poll+0x21b/0x230 [r6040]
[ 154.450195] [<c1043ab0>] ? __hrtimer_tasklet_trampoline+0x20/0x20
[ 154.450222] [<c1453633>] ? common_interrupt+0x33/0x40
[ 154.450247] [<c1043ab0>] ? __hrtimer_tasklet_trampoline+0x20/0x20
[ 154.450273] [<c1043b17>] ? __do_softirq+0x67/0x240
[ 154.450302] [<c1043ab0>] ? __hrtimer_tasklet_trampoline+0x20/0x20
[ 154.450327] [<c1003df5>] ? call_on_stack+0x45/0x50
[ 154.450340] <IRQ> [<c1043e55>] ? irq_exit+0x95/0xa0
[ 154.450378] [<c1453e02>] ? do_IRQ+0x42/0xc0
[ 154.450493] [<d0a68913>] ? scsi_request_fn+0x33/0x4b0 [scsi_mod]
[ 154.450590] [<d090ec60>] ? pcan_pci_cleanup+0x100/0x100 [pcan]
[ 154.450618] [<c1453633>] ? common_interrupt+0x33/0x40
[ 154.450663] [<d090ec60>] ? pcan_pci_cleanup+0x100/0x100 [pcan]
[ 154.450689] [<c1080000>] ? handle_edge_irq+0xe0/0x100
[ 154.450713] [<c107ef57>] ? __setup_irq+0x237/0x4e0
[ 154.450758] [<d090ec60>] ? pcan_pci_cleanup+0x100/0x100 [pcan]
[ 154.450783] [<c107f336>] ? request_threaded_irq+0xa6/0x130
[ 154.450826] [<d090ed28>] ? __pcan_pci_req_irq+0x58/0xb0 [pcan]
[ 154.450869] [<d090edcd>] ? pcan_pci_req_irq+0x4d/0x60 [pcan]
[ 154.450910] [<d0908329>] ? pcan_open_path+0x139/0x260 [pcan]
[ 154.450951] [<d09084f3>] ? pcan_open+0xa3/0x1a0 [pcan]
[ 154.450992] [<d0908549>] ? pcan_open+0xf9/0x1a0 [pcan]
[ 154.451022] [<c113a530>] ? mount_fs+0x190/0x190
[ 154.451049] [<c113ad14>] ? chrdev_open+0x84/0x140
[ 154.451087] [<c1134d0a>] ? do_dentry_open+0x19a/0x2d0
[ 154.451111] [<c113ac90>] ? cdev_put+0x30/0x30
[ 154.451138] [<c1134fca>] ? finish_open+0x2a/0x40
[ 154.451164] [<c11453f2>] ? do_last+0xae2/0xf90
[ 154.451191] [<c114ee00>] ? set_nlink+0x40/0x40
[ 154.451217] [<c1145cd4>] ? path_openat+0x434/0x5a0
[ 154.451246] [<c11465f1>] ? do_filp_open+0x31/0x90
[ 154.451282] [<c11364f3>] ? do_sys_open+0x113/0x220
[ 154.451311] [<c1136622>] ? SyS_open+0x22/0x30
[ 154.451338] [<c145308d>] ? syscall_call+0x10/0x10
[ 154.451355] handlers:
[ 154.451409] [<d090ec60>] pcan_pci_irqhandler [pcan]
[ 154.451445] Disabling IRQ #9

lsmod output:

Module Size Used by
8192cu 498104 0
mac80211 437441 0
cfg80211 366068 2 mac80211,8192cu
ftdi_sio 41955 0
usbserial 31597 1 ftdi_sio
rfkill 18387 1 cfg80211
evdev 17137 0
pcspkr 12531 0
serio_raw 12737 0
pcan 83138 0
autofs4 30769 2
ext4 442358 1
crc16 12327 1 ext4
mbcache 12892 1 ext4
jbd2 72871 1 ext4
sg 25643 0
sd_mod 43760 3
crc_t10dif 12399 1 sd_mod
crct10dif_generic 12517 1
crct10dif_common 12340 2 crct10dif_generic,crc_t10dif
ata_generic 12450 0
pata_rdc 12568 2
ohci_pci 12704 0
libata 161817 2 pata_rdc,ata_generic
ohci_hcd 42566 1 ohci_pci
ehci_pci 12464 0
ehci_hcd 64933 1 ehci_pci
scsi_mod 168109 3 sg,libata,sd_mod
usbcore 174991 7 usbserial,ohci_hcd,ohci_pci,ehci_hcd,ehci_pci,8192cu,ftdi_sio
r6040 17230 0
libphy 27572 1 r6040
usb_common 12484 1 usbcore

lspci -v output:

00:00.0 Host bridge: RDC Semiconductor, Inc. R6021 Host Bridge (rev 02) Flags: bus master, medium devsel, latency 0

00:03.0 Network controller: PEAK-System Technik GmbH PCAN-PCI CAN-Bus controller (rev 02)
Subsystem: PEAK-System Technik GmbH 2 Channel CAN Bus SJC1000
Flags: medium devsel, IRQ 9
Memory at fefd0000 (32-bit, non-prefetchable) [size=64K]
Memory at fefc0000 (32-bit, non-prefetchable) [size=64K]
Kernel driver in use: pcan

00:07.0 ISA bridge: RDC Semiconductor, Inc. R6031 ISA Bridge (rev 02)
Flags: bus master, ?? devsel, latency 0

00:08.0 Ethernet controller: RDC Semiconductor, Inc. R6040 MAC Controller
Flags: bus master, medium devsel, latency 128, IRQ 10
I/O ports at df00 [size=256]
Memory at fefff400 (32-bit, non-prefetchable) [size=256]
Expansion ROM at fefe0000 [disabled] [size=64K]
Kernel driver in use: r6040

00:0a.0 USB controller: RDC Semiconductor, Inc. R6060 USB 1.1 Controller (rev 12) (prog-if 10 [OHCI])
Flags: bus master, medium devsel, latency 128, IRQ 11
Memory at feffd000 (32-bit, non-prefetchable) [size=4K]
Kernel driver in use: ohci-pci

00:0a.1 USB controller: RDC Semiconductor, Inc. R6061 USB 2.0 Controller (rev 03) (prog-if 20 [EHCI])
Flags: bus master, medium devsel, latency 128, IRQ 11
Memory at fefff800 (32-bit, non-prefetchable) [size=256]
Kernel driver in use: ehci-pci

00:0b.0 USB controller: RDC Semiconductor, Inc. R6060 USB 1.1 Controller (rev 12) (prog-if 10 [OHCI])
Flags: bus master, medium devsel, latency 128, IRQ 11
Memory at feffe000 (32-bit, non-prefetchable) [size=4K]
Kernel driver in use: ohci-pci

00:0b.1 USB controller: RDC Semiconductor, Inc. R6061 USB 2.0 Controller (rev 03) (prog-if 20 [EHCI])
Flags: bus master, medium devsel, latency 128, IRQ 11
Memory at fefffc00 (32-bit, non-prefetchable) [size=256]
Kernel driver in use: ehci-pci

00:0c.0 IDE interface: RDC Semiconductor, Inc. Device 1011 (rev 01) (prog-if 8a [Master SecP PriP])
Subsystem: RDC Semiconductor, Inc. Device 1011
Flags: bus master, medium devsel, latency 64, IRQ 14
I/O ports at 01f0 [size=8]
I/O ports at 03f4
I/O ports at 0170 [size=8]
I/O ports at 0374
I/O ports at ef00 [size=16]
Kernel driver in use: pata_rdc

cat /proc/interrupts:

CPU0
0: 188192 XT-PIC-XT-PIC timer
1: 2 XT-PIC-XT-PIC i8042
2: 0 XT-PIC-XT-PIC cascade
8: 0 XT-PIC-XT-PIC rtc0
9: 100000 XT-PIC-XT-PIC
10: 4543 XT-PIC-XT-PIC eth1
11: 4073 XT-PIC-XT-PIC ehci_hcd:usb1, ehci_hcd:usb2, ohci_hcd:usb3, ohci_hcd:usb4
14: 1907 XT-PIC-XT-PIC pata_rdc
15: 76 XT-PIC-XT-PIC pata_rdc
NMI: 0 Non-maskable interrupts
LOC: 0 Local timer interrupts
SPU: 0 Spurious interrupts
PMI: 0 Performance monitoring interrupts
IWI: 0 IRQ work interrupts
RTR: 0 APIC ICR read retries
HYP: 0 Hypervisor callback interrupts
ERR: 1
MIS: 0

uname -r: 3.16.0-6-586

Tested with v8.5.1 and v8.7.0 drivers on Debian 8 32 bit. I tried the following, to no avail (troubleshooting guide for Ubuntu):

https://help.ubuntu.com/community/DebuggingIRQProblems

Said drivers were compiled with (only chardev):

make DNG=NO_DONGLE_SUPPORT USB_NO_USB_SUPPORT PCI=PCI_SUPPORT PCIEC=NO_PCIEC_SUPPORT ISA=NO_ISA_SUPPORT PCC=NO_PCARD_SUPPORT

I'm not sure what can I try next. We were using another mini-PCI CAN board prior to this whiich perhaps had not been completely uninstalled (several Google results point to some kind of hardware conflict, others refer to faulty firmware).

PeterBowman commented 2 years ago

I'm not planning on working on this anytime soon, but I'm definitely convinced that we'll drop the proprietary PCAN drivers in favor of SocketCAN: https://github.com/roboticslab-uc3m/yarp-devices/issues/251. So, WONTFIX I guess.