Specify data required for Views

[ebugden]

Currently, the UI doesn't show data requirements for views. So, it's not possible to determine what kinds of traces (ex. kernel and/or userspace) and what events (ex. sched switch) are required for a certain view.

Displaying view data requirements at the user's point of need (in the tool) is especially important for users who are managing tracing themselves (vs. using a standardized testing script) since it tells them how to produce the more complex analyses/views without them having to do frustrating guesswork.

Benefits:

• As a user who wants to have more precise control over what data I capture (less overhead), I need a way to discover what trace points are needed by various analyses, so I can learn how to modify my scripts, commands, or settings.
• Or as a user with no access to example tracing scripts, this allows me to view analyses quickly (without needing to digging through documentation, source code, asking devs).

---

Challenges

Providing this information to users is not trivial for several reasons:

• Server agnostic - The Trace Viewer client can be used with trace servers other than the Trace Compass Trace Server so view requirements need to come from the trace server and the way the information is displayed client-side needs to not be specific to a server
• Tracer agnostic - Trace points can have different names in different tracers (ex. Ftrace vs. LTTng)
• Trace format agnostic - Information about which events to enable is only relevant for trace formats that allow you to enable/disable certain events (some formats trace everything available all the time)
• Information about required events is distributed all over the place right now. There is no centralized place to find the events required for all analyses.

Resources

Some information about event requirements for views can be found here:

LTTng-utils Tracing Profiles

This resource is great! The profiles list the kernel and userspace (UST) events required for many core analyses (ex. Disk, Memory). Descriptions and other required bits are specified in the profiles. The profiles can also include each other. It's very clear and well organized.

Disk tracing profile

desc:
    The kernel events for disk analysis
kernel:
    - block_rq_complete
    - block_rq_insert
    - block_rq_issue
    - block_bio_frontmerge

Scheduler tracing profile

desc:
    The scheduler events of the kernel
kernel:
    - sched_process_exec
    - sched_process_exit
    - sched_process_fork
    - sched_process_free
    - sched_switch
    - sched_wakeup
    - sched_waking
    - sched_wakeup_new
    - sched_pi_setprio

General kernel tracing profile

desc:
    A subset of kernel events sufficient and necessary to make most Trace Compass analyses
includes:
    - sched
    - interrupt
    - network
    - timer
    - statedump_k
    - disk
kernel:
    - syscall
    - power_cpu_frequency

Trace Compass User Guide

Some of the view data requirements are in Trace Compass user guide, but it's incomplete, possibly outdated, requires digging, and requires that users know the tool's relationship to TC.

Show required trace formats?

Showing the trace types/formats required for the analysis could also be relevant, especially if we make it possible for users to explore the analyses that implemented, but not possible with the current trace type (See #285) to make the analyses more discoverable. Otherwise, users would have to read the documentation to discover the analysis or stumble onto the analysis by accident when they load the correct trace type.

It also allows users to understand which analyses require which kind of traces so they can intentionally collect the kinds of traces (ex. kernel, userspace, GPU) required to see an analysis. Otherwise, they can only stumble onto the analyses by accident if they happen to have the right collection of traces.

The solutions proposed below for displaying the required events could also be used for displaying the required trace types.

Show required events

• Quick fix - List required events at the end of each view description. If I understand correctly, according to @MatthewKhouzam this information is already available server side.
  • [ ] Curate view requirements
  • [ ] Transform requirements to raw text and concatenate to view description
• Tell user which events still need to be enabled. If an analysis is compatible with the trace type, but the events required to perform the analysis are not enabled (based on CTF trace metadata), these missing events can be listed.

Related: How should impossible analyses be presented #285

[ebugden]

Some questions for @bhufmann:

• Are there data requirements for views other than enabling the necessary trace events? For example, pointing to the source code or the executable that was traced.
• Does the list of events required for an analysis tend to be long (>5 events)? Are the names of trace events typically long? (This will impact how we can display the required events)

[bhufmann]

Some questions for @bhufmann:

* Are there data requirements for views other than enabling the necessary trace events? For example, pointing to the source code or the executable that was traced.

I most cases there are no other requirement than enabling the necessary trace events (and adding trace contexts). However, in some views symbol resolutions need to be done to translate PC addresses to source code location.

* Does the list of events required for an analysis tend to be long (>5 events)? Are the names of trace events typically long? (This will impact how we can display the required events)

There is not a generic answer for that. It depends on the view. For, LTTng thread status or resources view there are more than 5 events needed. See KernelAnalysisModule for the commented out code of required events. BTW, it's commented out because, this analysis module can be populated by LTTng, Perf or Ftrace traces that hook into the same kernel events, but have different event names, so that a static list of required event names won't work.

[ebugden]

The need to list the events required for the available views was highlighted by users in the user survey completed in November 2022.