freifunk-berlin / firmware

DEPRECATED: Build system for Berlin firmware. Please user the pinned falter-repos instead
https://berlin.freifunk.net
GNU General Public License v3.0
74 stars 34 forks source link

improve performance by distributing irqs over all CPUs #869

Open SvenRoederer opened 3 years ago

SvenRoederer commented 3 years ago

with the growing number of multi-CPU boards around it might be usefull to ensure that not a single CPU is doing all the work, while others are sleeping only. A pinning of interrupt to specific CPUs can be done manually, via /proc/ in the kernel. To automate this the package "irqbalance" exists. Reading the documents comes to the point the such pinning will only give a benefit for Multi-CPU systems having multiple IRQ-sources (e.g. ethernet(s), WiFi(s).

As I don't have such boards in reach, I can't tell which performance improvement this will give. I expect that the community can provide some data (/proc/interrupts, throughput-meassurements).

In addition the irqbalance daemon is not activated during package-install, which might require to create a "freifunk-berlin-smp-system" package to do the UCI-change, depend on irqbalance and probably do board specific changes.

SvenRoederer commented 3 years ago

This a dump od /proc/interrupts" from a ubnt ERX after some days uptime. There is only the ethernet-driver which is relevant for pinning. So these boards will likely not qualify for irq-pinning.

root@Taubenschlag:~# cat /proc/interrupts 
           CPU0       CPU1       CPU2       CPU3       
  8:  104453730  104453783  104453821  104453825  MIPS GIC Local   1  timer
  9:     113055          0          0          0  MIPS GIC  63  IPI call
 10:          0    3274564          0          0  MIPS GIC  64  IPI call
 11:          0          0    1291976          0  MIPS GIC  65  IPI call
 12:          0          0          0     866971  MIPS GIC  66  IPI call
 13:    1926360          0          0          0  MIPS GIC  67  IPI resched
 14:          0    5108401          0          0  MIPS GIC  68  IPI resched
 15:          0          0    1014618          0  MIPS GIC  69  IPI resched
 16:          0          0          0     737050  MIPS GIC  70  IPI resched
 17:          0          0          0          0  MIPS GIC  19  1e000600.gpio-bank0, 1e000600.gpio-bank1, 1e000600.gpio-bank2
 19:         12          0          0          0  MIPS GIC  33  ttyS0
 20:   14013701          0          0          0  MIPS GIC  10  1e100000.ethernet
 22:          0          0          0          0  1e000600.gpio  12  keys
ERR:          0

In contrast a Mikrotik RB750Gr3, there is in addition the USB and SDCard. For this board irq-pinning should improve, at least in theory.

root@Verklaerung-core:~# cat /proc/interrupts 
           CPU0       CPU1       CPU2       CPU3       
  8:  232833352  232833334  232833326  232833319  MIPS GIC Local   1  timer
  9:    2081009          0          0          0  MIPS GIC  63  IPI call
 10:          0    6174287          0          0  MIPS GIC  64  IPI call
 11:          0          0   34501949          0  MIPS GIC  65  IPI call
 12:          0          0          0    8520251  MIPS GIC  66  IPI call
 13:   11782796          0          0          0  MIPS GIC  67  IPI resched
 14:          0   13225794          0          0  MIPS GIC  68  IPI resched
 15:          0          0    9478747          0  MIPS GIC  69  IPI resched
 16:          0          0          0    4671113  MIPS GIC  70  IPI resched
 19:         12          0          0          0  MIPS GIC  33  ttyS0
 20:      16462          0          0          0  MIPS GIC  27  1e130000.sdhci
 21:          0          0          0          0  MIPS GIC  29  xhci-hcd:usb1
 22:  119039977          0          0          0  MIPS GIC  10  1e100000.ethernet
 23:          9          0          0          0  MIPS GIC  30  gsw
ERR:          1