search

clinthuffman / PAL

Performance Analysis of Logs (PAL) tool
MIT License
478 stars 101 forks source link

PAL doesn't add data for Process/Working Set Private to the htm report #35

Closed SDaniV closed 5 years ago

SDaniV commented 5 years ago

Hello Clint,

I encounter a problem that PAL doesn't generate data for Process/Working Set Private counter in htm report. When my Performance Counter contains many counters it looks like PAL just ignore Process/Working Set Private and generate the report without it. I created empty Performance Counter NewDataCollector, added only Process/Working Set Private there and I got such error message:

Overall progress... Status: Calculating counter statistics [very CPU intensive]..., PAL 2.8.1 Progress: 65%... Calculating counter statistics [very CPU intensive]...,

An error occurred on... $iPercentComplete = ConvertToDataType $(($iIndexOfJobs / $iTotalJobs) 100) 'integer' At C:\Program Files\PAL\PAL\PAL.ps1:2526 char:49 + ... ete = ConvertToDataType $(($iIndexOfJobs / $iTotalJobs) 100) 'integ ... + ~~~~~~~ Attempted to divide by zero.

SCRIPT ARGUMENTS: Log: C:\PerfLogs\Admin\NewDataCollector\BLACKBIRD_20190204-000001\DataCollector01.blg ThresholdFile: C:\Program Files\PAL\PAL\SystemOverview.xml AnalysisInterval: AUTO IsOutputHtml: $True IsOutputXml: $False HtmlOutputFileName: [LogFileName]_PALANALYSIS[DateTimeStamp].htm XmlOutputFileName: [LogFileName]_PALANALYSIS[DateTimeStamp].xml OutputDir: D:\PAL_results\IDLE_new_test AllCounterStats: $False NumberOfThreads: 4 IsLowPriority: $True DisplayReport: True

However, Working Set Private exists in the .blg file.

I use PAL v2.8.1 on Windows 10 64-bit.

clinthuffman commented 5 years ago

Hi SDaniV,

\Process()\Working Set – Private is not defined as a counter to analyze in the PAL threshold files, therefore, it is ignored. \Process()\Working Set is defined in the SystemOverview.xml threshold file and should show up in the PAL reports. Creating your own threshold file is certainly a good way to have it included in your reports, but only if you intend to analyze this counter often, then follow my articlehttps://blogs.technet.microsoft.com/clinth/2013/01/08/how-to-create-a-threshold-file-for-the-pal-tool/ on how to create a threshold file. Alternatively, you can force PAL to have the counter in the report as statistics only (no analysis) by checking the All Counter Stats checkbox on the Options tab.

I’m curious. Is there a reason why you are using \Process()\Working Set – Private instead of \Process()\Working Set? The difference between the two is Working Set includes both the private and public physical memory usage which is usually a trivial amount.

If you want \Process()\Working Set – Private to be analyzed the same as \Process()\Working Set – Private, then I can modify a threshold file for you to do that.

Thank you, Clint Huffman The “PALhttp://github.com/clinthuffman/PAL” guy

From: SDaniV notifications@github.com Sent: Monday, February 4, 2019 2:32:00 AM To: clinthuffman/PAL Cc: Subscribed Subject: [clinthuffman/PAL] PAL doesn't add data for Process/Working Set Private to the htm report (#35)

Hello Clint,

I encounter a problem that PAL doesn't generate data for Process/Working Set Private counter in htm report. When my Performance Counter contains many counters it looks like PAL just ignore Process/Working Set Private and generate the report without it. I created empty Performance Counter NewDataCollector, added only Process/Working Set Private there and I got such error message:

Overall progress... Status: Calculating counter statistics [very CPU intensive]..., PAL 2.8.1 Progress: 65%... Calculating counter statistics [very CPU intensive]...,

An error occurred on... $iPercentComplete = ConvertToDataType $(($iIndexOfJobs / $iTotalJobs) 100) 'integer' At C:\Program Files\PAL\PAL\PAL.ps1:2526 char:49 + ... ete = ConvertToDataType $(($iIndexOfJobs / $iTotalJobs) 100) 'integ ... + ~~~~~~~ Attempted to divide by zero.

SCRIPT ARGUMENTS: Log: C:\PerfLogs\Admin\NewDataCollector\BLACKBIRD_20190204-000001\DataCollector01.blg ThresholdFile: C:\Program Files\PAL\PAL\SystemOverview.xml AnalysisInterval: AUTO IsOutputHtml: $True IsOutputXml: $False HtmlOutputFileName: [LogFileName]PAL_ANALYSIS[DateTimeStamp].htm XmlOutputFileName: [LogFileName]PAL_ANALYSIS[DateTimeStamp].xml OutputDir: D:\PAL_results\IDLE_new_test AllCounterStats: $False NumberOfThreads: 4 IsLowPriority: $True DisplayReport: True

However, Working Set Private exists in the .blg file.

I use PAL v2.8.1 on Windows 10 64-bit.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fclinthuffman%2FPAL%2Fissues%2F35&data=02%7C01%7C%7C467a180c8a5d4f4f800e08d68a8bff54%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636848731213818288&sdata=kayAvIEpiiqn0VFTTJEuFQBbald97VzN6e2kwfOXZEk%3D&reserved=0, or mute the threadhttps://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAS48VBWKxB7_bNEWPMVJ6xgQehFwtJf6ks5vKAwggaJpZM4ag6_Q&data=02%7C01%7C%7C467a180c8a5d4f4f800e08d68a8bff54%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636848731213828293&sdata=OQqR%2B4YW%2FkojrSWdB4ydvoo0RbqjuaAroIEOGzDvo6E%3D&reserved=0.

jeffstokes72 commented 5 years ago

I was curious on the applicability of this as well, I have to admit...

On Mon, Feb 4, 2019 at 6:26 PM Clint Huffman notifications@github.com wrote:

Hi SDaniV,

\Process()\Working Set – Private is not defined as a counter to analyze in the PAL threshold files, therefore, it is ignored. \Process()\Working Set is defined in the SystemOverview.xml threshold file and should show up in the PAL reports. Creating your own threshold file is certainly a good way to have it included in your reports, but only if you intend to analyze this counter often, then follow my article< https://blogs.technet.microsoft.com/clinth/2013/01/08/how-to-create-a-threshold-file-for-the-pal-tool/> on how to create a threshold file. Alternatively, you can force PAL to have the counter in the report as statistics only (no analysis) by checking the All Counter Stats checkbox on the Options tab.

I’m curious. Is there a reason why you are using \Process()\Working Set – Private instead of \Process()\Working Set? The difference between the two is Working Set includes both the private and public physical memory usage which is usually a trivial amount.

If you want \Process()\Working Set – Private to be analyzed the same as \Process()\Working Set – Private, then I can modify a threshold file for you to do that.

Thank you, Clint Huffman The “PALhttp://github.com/clinthuffman/PAL” guy

From: SDaniV notifications@github.com Sent: Monday, February 4, 2019 2:32:00 AM To: clinthuffman/PAL Cc: Subscribed Subject: [clinthuffman/PAL] PAL doesn't add data for Process/Working Set Private to the htm report (#35)

Hello Clint,

I encounter a problem that PAL doesn't generate data for Process/Working Set Private counter in htm report. When my Performance Counter contains many counters it looks like PAL just ignore Process/Working Set Private and generate the report without it. I created empty Performance Counter NewDataCollector, added only Process/Working Set Private there and I got such error message:

Overall progress... Status: Calculating counter statistics [very CPU intensive]..., PAL 2.8.1 Progress: 65%... Calculating counter statistics [very CPU intensive]...,

An error occurred on... $iPercentComplete = ConvertToDataType $(($iIndexOfJobs / $iTotalJobs) 100) 'integer' At C:\Program Files\PAL\PAL\PAL.ps1:2526 char:49 + ... ete = ConvertToDataType $(($iIndexOfJobs / $iTotalJobs) 100) 'integ ... +


Attempted to divide by zero.

SCRIPT ARGUMENTS:
Log:
C:\PerfLogs\Admin\NewDataCollector\BLACKBIRD_20190204-000001\DataCollector01.blg
ThresholdFile: C:\Program Files\PAL\PAL\SystemOverview.xml
AnalysisInterval: AUTO
IsOutputHtml: $True
IsOutputXml: $False
HtmlOutputFileName: [LogFileName]PAL_ANALYSIS[DateTimeStamp].htm
XmlOutputFileName: [LogFileName]PAL_ANALYSIS[DateTimeStamp].xml
OutputDir: D:\PAL_results\IDLE_new_test
AllCounterStats: $False
NumberOfThreads: 4
IsLowPriority: $True
DisplayReport: True

However, Working Set Private exists in the .blg file.

I use PAL v2.8.1 on Windows 10 64-bit.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub<
https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fclinthuffman%2FPAL%2Fissues%2F35&data=02%7C01%7C%7C467a180c8a5d4f4f800e08d68a8bff54%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636848731213818288&sdata=kayAvIEpiiqn0VFTTJEuFQBbald97VzN6e2kwfOXZEk%3D&reserved=0>,
or mute the thread<
https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAS48VBWKxB7_bNEWPMVJ6xgQehFwtJf6ks5vKAwggaJpZM4ag6_Q&data=02%7C01%7C%7C467a180c8a5d4f4f800e08d68a8bff54%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636848731213828293&sdata=OQqR%2B4YW%2FkojrSWdB4ydvoo0RbqjuaAroIEOGzDvo6E%3D&reserved=0
>.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<https://github.com/clinthuffman/PAL/issues/35#issuecomment-460454700>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AIrl-XyLJZ0EvlBbKe9rbYc3hIrJl2_9ks5vKMG6gaJpZM4ag6_Q>
.
SDaniV commented 5 years ago

Hi Clint,

I'm sorry that it took so long to get back to you.

I'm just interested in the statistics to compare Working Set with Working Set - Private for a specific process and add it to .htm report. I've been trying to enable All Counter Stats checkbox but the counter wasn't included to the report..

Is it possible to modify the current file without creating a new one? When I go to Threshold file -> Edit -> New ...(add counter) -> Update Analysis -> Save I'm getting error message.

Also, I tried to create a new one as you described in the article but for me not everything is clear: Should I modify the tool to use it instead of SystemOverview.xml or there is some easiest way :)

Thank you

clinthuffman commented 5 years ago

Likely you can’t save the file because it is in a secure location. Try copying the PAL installation folder to your desktop and then running PALWizard.exe from there, then try editing and saving the threshold file.

There again, if you are just comparing two counters, then viewing them in Perfmon sounds easier.

Thank you,

From: SDaniV notifications@github.com Sent: Thursday, February 7, 2019 4:51:23 AM To: clinthuffman/PAL Cc: Clint Huffman; Comment Subject: Re: [clinthuffman/PAL] PAL doesn't add data for Process/Working Set Private to the htm report (#35)

Hi Clint,

I'm sorry that it took so long to get back to you.

I'm just interested in the statistics to compare Working Set with Working Set - Private for a specific process and add it to .htm report. I've been trying to enable All Counter Stats checkbox but the counter wasn't included to the report..

Is it possible to modify the current file without creating a new one? When I go to Threshold file -> Edit -> New ...(add counter) -> Update Analysis -> Save I'm getting error message.

Also, I tried to create a new one as you described in the article but for me not everything is clear: Should I modify the tool to use it instead of SystemOverview.xml or there is some easiest way :)

Thank you

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fclinthuffman%2FPAL%2Fissues%2F35%23issuecomment-461406677&data=02%7C01%7C%7C1df4cfc7f7c549f1d74408d68cfaf936%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636851406878420965&sdata=NmJQ%2FQDbDtTzo9KubS9c1AiV9sT%2Bj5CZzE2%2BeOcvaUo%3D&reserved=0, or mute the threadhttps://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAS48VHFgoxdIZjZ6aZ3dMLovwy4uZQP0ks5vLCFLgaJpZM4ag6_Q&data=02%7C01%7C%7C1df4cfc7f7c549f1d74408d68cfaf936%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636851406878430976&sdata=gPK4z6esXtJWSBoz2U5AJhaXkOIvWr5JiO%2FtIKvVyaU%3D&reserved=0.

SDaniV commented 5 years ago

Hi Clint,

I'm terribly sorry, I tried your suggestion only now. When I try to edit System Overview I get Working Set - Private to the report but without the chart and for some reason, half of the other charts break. At the same time, when I try to edit Quick System Overview everything works fine but in such case, I can't get another process counters...

Thank you for trying to help me!

clinthuffman commented 5 years ago

SystemOverview.xml.txt Hi SDaniV, I updated SystemOverview.xml with the Working Set - Private counter. It is statistics only - no thresholds, but it works. I attached it to this response.

Rename SystemOverview.xml.txt (attached) to SystemOverview.xml and then replace the same named file in the PAL installation folder, then try it out.

SDaniV commented 5 years ago

Hi Clint,

This is exactly what I was looking for. Thank you very much!