google / UIforETW

User interface for recording and managing ETW traces
https://randomascii.wordpress.com/2015/04/14/uiforetw-windows-performance-made-easier/
Apache License 2.0
1.55k stars 201 forks source link

Could UIForETW help in a case of getting only the necessary CPU/Process data? #142

Closed ghost closed 4 years ago

ghost commented 4 years ago

Hello

I have a general question, just so I don't waste time digging deeper only to figure out it's not what I expected, but I'm doing my homework of trying to listen and do as much of reading and watching of various WPT talks and explanations.

Now the WPA stuff I'm normally doing is general game troubleshooting and beta testing to provide feedback, but there's another project which is my own with this stuff that's unrelated but coincides to be in exactly the same field. Seems like for this idea I would need to develop my own .ETL/ETW parsing utility with which I could derive and calculate things that you can't do with WPA.

It's about CPU Usage and thread stuff and there is this kown fact from the docs and the communities from what understand that the "CPU Precise can't see what threads are doing between switching" and that "CPU Sampled can't see the stuff CPU Precise can" which kinda complicates things for me going forward, which one would I need, or both. However from my research I've figured out which kind of data I think I would need for this, and only parse that with my utility, still using the WPR or UIForETW sources.

However, by default WPR even with just the "CPU Usage" resource cagegory, captures huge amounts of data (Verbose) for all processes that also becomes even bigger when exported to CSV format, which is what I would feed into my custom parsing utility for simplicity, I don't want to spend time on coding support for some special formats. I'm not a real programmer, just trying to create a demo to prove a point on an idea I think may be very interesting, at least for the technical people.

Perhaps there is something that could help me get up to speed out there, I even saw some parsing stuff in the UIForETW folder, but I have no Python experience and kinda want to stick with C# which I know best, I'd even do this in C++ if I had the experience.

I've seen that these loggers are system wide and capture all processes, but I've perhaps heard wpaexporter could come in handy by extracting only thing things I need, for a specific process, to get filesizes down, but I'm just right now investigating that deeper if that's really the case. I only need CPU ID, Switch times and a few thread details stuff, no stacks or any DLL or any other stuff.

The question is would UIForETW help me in such a case, for me to use it instead of WPR and then give the output to the wpaexporter.

Thanks for suggestions.

randomascii commented 4 years ago

ETW by its nature records information across processes and UIforETW will not be doing anything to fix that.

The question is would UIForETW help and use it instead of WPR

No. The details vary but for your purposes the information recorded is essentially identical.

A few thoughts for your project:

CPU Usage (Sampled) is good for figuring out what the CPU is doing CPU Usage (Precise) is good for figuring out why the CPU is blocked from doing what you want it to do

Which is most appropriate depends on what your performance problem is and getting a generic parsing utility to analyze that can be tricky or even impossible.

Doing bulk trace analysis using .csv files is the wrong path to go down. There are better ways available now. See this post for details: https://randomascii.wordpress.com/2020/01/05/bulk-etw-trace-analysis-in-c/

For all other ETW questions I recommend this page: https://randomascii.wordpress.com/2015/09/24/etw-central/

But because I'm not a real programmer...

Good luck. ETW analysis is tricky and, especially if you want to avoid confidently coming to wrong conclusions, often requires deep knowledge of what the operating system is doing.

ghost commented 4 years ago

Oh, thanks for a speedy reply, yeah I wasn't sure where to ask, I'll go there for more if I need in the future.

Re reading the xperf cpu sheduling article, ... seems like the "this presentation" link is broken.

Yeah I had second thoughts about CSV ...

I do actually plan to join a real programming course so I'll probably be fine :)

randomascii commented 4 years ago

I deleted the bad link. Thanks for pointing that out.