search

AmpereComputing / ampere-lts-kernel

Linux 5.4 and 5.10 Longterm kernel (LTS) with Ampere patches
21 stars 17 forks source link

refactor patch: perf/smmuv3: Allow sharing MMIO registers with the SMMU driver #152

Open adamliyi opened 2 years ago

adamliyi commented 2 years ago

In linux-5.4.y kernel, we backported two smmu related patches:

  1. https://github.com/AmpereComputing/ampere-lts-kernel/commit/779bed7502d9d2c77c3a70521d4baf9a3b8f4669: perf/smmuv3: Allow sharing MMIO registers with the SMMU driver. This patch is not upstream, original patch from: https://github.com/AmpereComputing/ampere-centos-kernel/commit/1164351241b52b3c8ffab4e29de5f93a9abe4acf
  2. https://github.com/AmpereComputing/ampere-lts-kernel/commit/87957b27f791fb4ea044264e0342a3da1b711dfa: iommu/arm-smmu-v3: Don't reserve implementation defined register space. This patch is in upstream 5.8 kernel: https://github.com/torvalds/linux/commit/52f3fab0067d6fa9e99c1b7f63265dd48ca76046

The purpose of these two patches are the same, some iomem region of SMMU and SMMU-PMU (i.e pmcg) overlapped:

$ cat /proc/iomem
23ffe0000000-23ffe001ffff : arm-smmu-v3.2.auto
  23ffe0002000-23ffe0002fff : arm-smmu-v3-pmcg.36.auto <------- overlapped region
  23ffe0012000-23ffe0012fff : arm-smmu-v3-pmcg.36.auto <------- overlapped region
23ffe0042000-23ffe0042fff : arm-smmu-v3-pmcg.26.auto
23ffe0052000-23ffe0052fff : arm-smmu-v3-pmcg.26.auto
23ffe0062000-23ffe0062fff : arm-smmu-v3-pmcg.27.auto
23ffe0072000-23ffe0072fff : arm-smmu-v3-pmcg.27.auto

if both smmuv3 driver and smmuv3-pmu try to request the overlapped region, there may be error message like

see bug #73682
[   22.496214] arm-smmu-v3 arm-smmu-v3.3.auto: can't request region for resource [mem 0x27ffe0000000-0x27ffe001ffff]
[   22.516904] arm-smmu-v3: probe of arm-smmu-v3.3.auto failed with error -16
[   22.516904] arm-smmu-v3: probe of arm-smmu-v3.3.auto failed with probe of arm-smmu-v3.7.auto failed with error -16
[   22.713868] arm-smmu-v3 arm-smmu-v3.8.auto: option mask 0x0
[   22.713868] arm-smmu-v3 arm-smmu-v3.8.auto: option mask 0x0
[   22.725090] arm-smmu-v3 arm-smmu-v3.8.auto: can't request region for resource [mem 0x67ffe0000000-0x67ffe001ffff]
[   22.725090] arm-smmu-v3 arm-smmu-v3.8.auto: can't request region for resource [mem 0x67ffe0000000-0x67ffe0 '

Since both patches tries to fix the same issue, we just need to keep the upstream backporting, i.e: 'iommu/arm-smmu-v3: Don't reserve implementation defined register space'

adamliyi commented 2 years ago

On 5.15.y, drop: https://github.com/AmpereComputing/ampere-lts-kernel/commit/56ca42aa518d063259f6bf44cb8b0f3acc859538

Rebase instead of revert, to keep the commit message clean.

drop 56ca42aa518d0 perf/smmuv3: Allow sharing MMIO registers with the SMMU driver
pick 322604bf5371b arm64: enable altra sensors and leds
adamliyi commented 2 years ago

On 5.4.y, revert https://github.com/AmpereComputing/ampere-lts-kernel/commit/779bed7502d9d2c77c3a70521d4baf9a3b8f4669: perf/smmuv3: Allow sharing MMIO registers with the SMMU driver

adamliyi commented 2 years ago

One remaining issue: Accessing SMMU PMU in overlapped region, looks does not work properly:

23ffe0000000-23ffe001ffff : arm-smmu-v3.2.auto
  23ffe0000000-23ffe0000dff : arm-smmu-v3.2.auto
  23ffe0002000-23ffe0002fff : arm-smmu-v3-pmcg.36.auto <---------------------------------- pmcg.36 in overlapped region
    23ffe0002000-23ffe0002fff : arm-smmu-v3-pmcg.36.auto arm-smmu-v3-pmcg.36.auto
  23ffe0010000-23ffe0010dff : arm-smmu-v3.2.auto
  23ffe0012000-23ffe0012fff : arm-smmu-v3-pmcg.36.auto
    23ffe0012000-23ffe0012fff : arm-smmu-v3-pmcg.36.auto arm-smmu-v3-pmcg.36.auto
23ffe0042000-23ffe0042fff : arm-smmu-v3-pmcg.26.auto  <---------------- pmcg.26 NOT in overlapped region
  23ffe0042000-23ffe0042fff : arm-smmu-v3-pmcg.26.auto arm-smmu-v3-pmcg.26.auto
# perf stat -e smmuv3_pmcg_23ffe0002/cycles/ -I 1000
#           time             counts unit events
     1.001071197         2147483649      smmuv3_pmcg_23ffe0002/cycles/                             
     2.002265879                  0      smmuv3_pmcg_23ffe0002/cycles/            <--------- zero value                   
     3.003336435                  0      smmuv3_pmcg_23ffe0002/cycles/
# perf stat -e smmuv3_pmcg_23ffe0042/cycles/ -I 1000
#           time             counts unit events
     1.001072197          500515065      smmuv3_pmcg_23ffe0042/cycles/                             
     2.002267399          500576281      smmuv3_pmcg_23ffe0042/cycles/           <---------- expected non-zero value                  
     3.003353396          500522773      smmuv3_pmcg_23ffe0042/cycles/                             
     4.004435953          500521953      smmuv3_pmcg_23ffe0042/cycles/