What PMUs would be useful to support in simpleperf

[appujee]

cc: @pirama-arumuga-nainar @yabinc

[yabinc]

I found riscv cpu pmu support in https://github.com/torvalds/linux/blob/master/drivers/perf/riscv_pmu_sbi.c#L78. It includes the most common hardware perf events like cpu-cycles, instructions, cache, and stall. They can be automatically supported in simpleperf. There seem to also have some raw events (not mapped by the kernel) in https://github.com/torvalds/linux/blob/master/arch/riscv/include/asm/sbi.h#L164. For ARM raw events, simpleperf provides a mapping from raw event names to raw event numbers, as in https://android.googlesource.com/platform/system/extras/+/master/simpleperf/event_type_table.h#199). We can do something similar to riscv raw events. Another thing we can do is to enforce riscv cpu pmu support in CtsSimpleperfTestCases, as what we did for arm cpu pmu.

[atishp04]

FYI: Qemu riscv virt machine only supports very few perf events. https://github.com/qemu/qemu/blob/master/target/riscv/pmu.c#L51

I am not sure what environment you are running. If it is anything other than riscv virt machine, you probably need to update the device tree node enable those events for firmware to consume it. https://github.com/qemu/qemu/blob/master/target/riscv/pmu.c#L77

We are working on the counter delegation ISA extensions in the Qemu. Once that is available, it can be configured via json files in perf though. I guess EVENT_TYPE_TABLE_ENTRY in simpleperf provides the same infrastructure.

I didn't see any option to provide event to counter mapping in simple perf though. Does it exist today ? My guess it doesn't as it was not required for ARM64 which supports all-to-all mapping. I would be happy to be proven wrong though.

[SiFiveHolland]

For some examples of which "raw" events are available on existing hardware, several mapping files have been submitted to the mailing list (none of them appear to have been merged): https://lore.kernel.org/linux-riscv/?q=%22pmu-events%22

[yabinc]

Simpleperf doesn't support Json configuration files for each CPU core type. Instead, it adds all possible events in EVENT_TYPE_TABLE_ENTRY, and test if they are available at runtime. I feel now this isn't precise enough. I plan to also support Json configurations in simpleperf. We can add RISCV configurations after that.

[captain5050]

At the RISC-V summit last week and at Linux plumbers this week there was a repeated ask that simpleperf and Linux's perf tool be unforked. This was a concern particularly expressed by developers. An example issue is that while Android supported BIG.little for around 10 years, the kernel PMU driver on ARM was broken and fixed just in Linux 6.6 - this was prompted by Linux perf tool fixes being done for Intel's hybrid. A straightforward backport should happen to 5 series kernels by way of a fixes tag.

Historically, and I can't imagine it changing, kernel and PMU developers haven't cared about the simpleperf case. The issue this gives us on RISC-V is that kernel patches can lack testing as the intended use is simpleperf.

The bar for events being added to Linux's perf tool isn't particularly high and is generally gated by myself. We have two RISC-V architectures in perf-tools-next which merges to linux-next (and from there into Linus' tree although there may be some stalling related to merge windows).

There are proposed changes to Linux's perf json format and tooling, json files in the tree will be updated. Tools generating the data (e.g. Intel's) will also be updated, but we can only be reasonable to things we know about and maintenance of simpleperf is out-of-scope.

The PMU driver code in the Linux kernel is being looked at differently by Atish at Rivos than for existing platforms like ARM and Intel. The particular problem is that there is a lack of standardization of PMU events in RISC-V and so there is a desire to make legacy event discovery work differently outside of the kernel/PMU module. This obviously needs fixing in the Linux tree, but it hasn't yet, and so the problem is more than just having simpleperf catch up with Linux perf. The problem solver isn't yet even looking at the BIG.little case.

An area where RISC-V is particularly lacking in support is with uncore events - I'm not aware of any PMUs supporting this. There are also two proposed instruction tracing formats, think intel-pt or ARM coresight, but RISC-V support here is targeting the embedded rather than Linux use-case. This will need fixing but it'd be nice not to be duplicating the work.

[enh-google]

At the RISC-V summit last week and at Linux plumbers this week there was a repeated ask that simpleperf and Linux's perf tool be unforked.

i assume you mean specifically just the supported events? (rather than the linux kernel folks looking forward to supporting Windows hosts :-) )

if you can get the linux list in a format we can use with a license we can use, that sgtm. (i'm assuming the latter conjunct will be the more challenging one.)

[captain5050]

i assume you mean specifically just the supported events? (rather than the linux kernel folks looking forward to supporting Windows hosts :-) )

I don't think simpleperf works with events in Windows, perf may via some WSL magic ¯_(ツ)_/¯ we need to support Windows as a guest operating system. We've been talking about putting things like WASM builds into the Linux perf build as it is vanilla C code if you strip out things like BPF. I imagine Windows would be a simpler target.

if you can get the linux list in a format we can use with a license we can use, that sgtm. (i'm assuming the latter conjunct will be the more challenging one.)

Perf code is largely GPLv2 (as with Android shipping the Linux kernel, JDK libraries, etc. - the Linux perf build explicitly makes any GPLv3 dependencies opt-in as GPLv2 is incompatible with GPLv3 and distributions can't ship it). New code in Linux perf is being added with LGPL-2.1 OR BSD-2-Clause which is the license used by libbpf. Android uses BSD as a license in places like bionic. As an example, it makes sense to use perf's event parsing with say a project using perf events with BPF and licenses shouldn't inhibit this.

[enh-google]

yeah, as long as the bit we should share is in 2BSD code, that should be fine.

[captain5050]

yeah, as long as the bit we should share is in 2BSD code, that should be fine.

I think 3rd parties would prefer just shipping Linux perf and migrating the unwinding and other features of simpleperf back to it. Android already ships GPLv2 in many places like:

https://cs.android.com/android/platform/superproject/main/+/main:libcore/ojluni/src/main/java/java/lang/Class.java;l=7

[captain5050]

yeah, as long as the bit we should share is in 2BSD code, that should be fine.

I think 3rd parties would prefer just shipping Linux perf and migrating the unwinding and other features of simpleperf back to it.

Fwiw, this is what 3rd parties explicitly requested to me - so less of a thought.