search

dotnet / corefx-tools

Place to include various misc tools for .NET and .NET Core.
MIT License
58 stars 41 forks source link

'cpu-clock' event do not have CPU attribute set #84

Open brianrob opened 6 years ago

brianrob commented 6 years ago

Some configurations result in the CPU attribute not being set on cpu-clock events. We get an explicit failure on calls to 'perf script' when this happens because we ask for the CPU number.

Consider removing this or looking for a way to optionally get the CPU number.

brianrob commented 6 years ago

Example issue: https://github.com/dotnet/coreclr/issues/18452

paraspatidar commented 5 years ago

FYI , apart from kubernetes , same behavior is observed on Ubuntu VM on azure. when running that command manually : /usr/bin/perf script -i perf.data.merged -f comm,pid,tid,cpu,time,event,ip,sym,dso,trace > perf.data.txt : output : Samples for 'cpu-clock' event do not have CPU attribute set

Note : this happens when we do perfcollect for specific pid , other wise , for normal perfcollect , it works fine and generates CPU traces on Ubuntu.

OS : NAME="Ubuntu" VERSION="18.04.1 LTS (Bionic Beaver)" ID=ubuntu ID_LIKE=debian PRETTY_NAME="Ubuntu 18.04.1 LTS" VERSION_ID="18.04" HOME_URL="https://www.ubuntu.com/" SUPPORT_URL="https://help.ubuntu.com/" BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" VERSION_CODENAME=bionic UBUNTU_CODENAME=bionic

uname -r : 4.18.0-1011-azure Dotnet core version : 2.2.2 / 2.1

lathiat commented 4 years ago

I found this issue on Google when researching a similar error but in a different software suite.. sharing a few notes here in case someone comes back to this ticket:

I believe the reason this happens is that if you are on metal with hardware performance counters, perf uses the 'cycles' event that has the CPU number. If you are on a VM without hardware performance counters (which is most of the time, particularly in public clouds due to security reasons, I think some AWS instance types where you have dedicated hardware access might give you some performance counters even in a VM), then it uses the software 'cpu-clock' event which does not have this field.

I didn't get to figuring out why in some configurations using all-CPU versus per-process mode seems to result in the CPU field appearing or disappearing.

When using "perf script"'s "-F" field specifier you can specify different fields for "software", "hardware" and "trace" events. Unfortunately if you specify a format for hardware events and there are no such in the file then it errors out. So it seems we can't ask "perf script" to handle this for us and likely some kind of software detection is going to be required.

Hope it helps someone...