search

Azure / batch-insights

One liners to enable Application insights on azure batch
MIT License
24 stars 16 forks source link

No Metrics Showing #39

Closed SharmainePopov closed 5 years ago

SharmainePopov commented 5 years ago

Does not show performance graphs. Just the tabs for them.

batchmetrics batchsettings

notedstats stderr

timotheeguerin commented 5 years ago

Looks like you have a typo in the BATCH_INSIGHTS_DOWNLOAD_UR env variable(missing the L

SharmainePopov commented 5 years ago

Hi typo fixed but still no graphs.

Get Outlook for Androidhttps://aka.ms/ghei36

From: Timothee Guerin notifications@github.com Sent: Tuesday, June 11, 2019 11:06:30 AM To: Azure/batch-insights Cc: Sharmaine Popov; Author Subject: Re: [Azure/batch-insights] No Metrics Showing (#39)

Looks like you have a typo in the BATCH_INSIGHTS_DOWNLOAD_UR env variable(missing the L

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FAzure%2Fbatch-insights%2Fissues%2F39%3Femail_source%3Dnotifications%26email_token%3DAIIVNWRBHV4VATH4OV275ILPZ65PNA5CNFSM4HW2AXFKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXNOGHA%23issuecomment-500884252&data=02%7C01%7CSharmaine.Popov%40microsoft.com%7Cadd2fc12a79644471e4708d6ee7e62c9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636958623926293780&sdata=wYJUu1zQjkQ%2BvPNKPuhO9wd4PY9YUFj%2FJrWnC9oewIY%3D&reserved=0, or mute the threadhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAIIVNWSPCLB2PRG4VWIYDETPZ65PNANCNFSM4HW2AXFA&data=02%7C01%7CSharmaine.Popov%40microsoft.com%7Cadd2fc12a79644471e4708d6ee7e62c9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636958623926303780&sdata=1CsNEosEgjKogYGU94xQHDRHB2dyHFsO94%2B%2FUgciS0k%3D&reserved=0.

SharmainePopov commented 5 years ago

...also same errors.

Get Outlook for Androidhttps://aka.ms/ghei36

From: Timothee Guerin notifications@github.com Sent: Tuesday, June 11, 2019 11:06:30 AM To: Azure/batch-insights Cc: Sharmaine Popov; Author Subject: Re: [Azure/batch-insights] No Metrics Showing (#39)

Looks like you have a typo in the BATCH_INSIGHTS_DOWNLOAD_UR env variable(missing the L

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FAzure%2Fbatch-insights%2Fissues%2F39%3Femail_source%3Dnotifications%26email_token%3DAIIVNWRBHV4VATH4OV275ILPZ65PNA5CNFSM4HW2AXFKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXNOGHA%23issuecomment-500884252&data=02%7C01%7CSharmaine.Popov%40microsoft.com%7Cadd2fc12a79644471e4708d6ee7e62c9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636958623926293780&sdata=wYJUu1zQjkQ%2BvPNKPuhO9wd4PY9YUFj%2FJrWnC9oewIY%3D&reserved=0, or mute the threadhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAIIVNWSPCLB2PRG4VWIYDETPZ65PNANCNFSM4HW2AXFA&data=02%7C01%7CSharmaine.Popov%40microsoft.com%7Cadd2fc12a79644471e4708d6ee7e62c9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636958623926303780&sdata=1CsNEosEgjKogYGU94xQHDRHB2dyHFsO94%2B%2FUgciS0k%3D&reserved=0.

timotheeguerin commented 5 years ago

hhm, Same error in stderr.txt as well? I guess you rebooted the node(s) after the change?

SharmainePopov commented 5 years ago

Oh the reboot. Tripped over that before. Ok let me do that.

Sharmaine Popov Cloud Solution Architect |Microsoft Canada Inc. | sharmaine.popov@microsoft.commailto:sharmaine.popov@microsoft.com | 416-799-0807

From: Timothee Guerin notifications@github.com Sent: Tuesday, June 11, 2019 12:34 PM To: Azure/batch-insights batch-insights@noreply.github.com Cc: Sharmaine Popov Sharmaine.Popov@microsoft.com; Author author@noreply.github.com Subject: Re: [Azure/batch-insights] No Metrics Showing (#39)

hhm, Same error in stderr.txt as well? I guess you rebooted the node(s) after the change?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FAzure%2Fbatch-insights%2Fissues%2F39%3Femail_source%3Dnotifications%26email_token%3DAIIVNWWHFGVGKHN3WG4BJCLPZ7HWBA5CNFSM4HW2AXFKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXNXOZY%23issuecomment-500922215&data=02%7C01%7CSharmaine.Popov%40microsoft.com%7Cf3cc331e286d4e80d5c708d6ee8a8dc2%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636958676182457686&sdata=1%2BjM93ZdorcuYarirTD1KjrLjFVUWmz0%2FUzTEn1NE3Q%3D&reserved=0, or mute the threadhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAIIVNWWGSBE3FRTVZMVR5U3PZ7HWBANCNFSM4HW2AXFA&data=02%7C01%7CSharmaine.Popov%40microsoft.com%7Cf3cc331e286d4e80d5c708d6ee8a8dc2%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636958676182467693&sdata=ZkHZX272cI4s5aAh0NhMSfPmZTPykqu3hFIGsO623N8%3D&reserved=0.

timotheeguerin commented 5 years ago

In Batch Explorer after you update the start task you get a notification that lets you reboot all the nodes at once too.

SharmainePopov commented 5 years ago

? I have never noticed that.

Sharmaine Popov Cloud Solution Architect |Microsoft Canada Inc. | sharmaine.popov@microsoft.commailto:sharmaine.popov@microsoft.com | 416-799-0807

From: Timothee Guerin notifications@github.com Sent: Tuesday, June 11, 2019 12:36 PM To: Azure/batch-insights batch-insights@noreply.github.com Cc: Sharmaine Popov Sharmaine.Popov@microsoft.com; Author author@noreply.github.com Subject: Re: [Azure/batch-insights] No Metrics Showing (#39)

In Batch Explorer after you update the start task you get a notification that lets you reboot all the nodes at once too.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FAzure%2Fbatch-insights%2Fissues%2F39%3Femail_source%3Dnotifications%26email_token%3DAIIVNWWRPZZ5J5NXADBJG7LPZ7H77A5CNFSM4HW2AXFKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXNXXTA%23issuecomment-500923340&data=02%7C01%7CSharmaine.Popov%40microsoft.com%7Cb0413dac1d684ba622b108d6ee8aec93%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636958677780173497&sdata=pcpL1sjpEmALqrhM04wWxT4LSEkUA1l7xxGfrE3x1g0%3D&reserved=0, or mute the threadhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAIIVNWSGL3PAZ2NXXFVTCTDPZ7H77ANCNFSM4HW2AXFA&data=02%7C01%7CSharmaine.Popov%40microsoft.com%7Cb0413dac1d684ba622b108d6ee8aec93%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636958677780173497&sdata=uHML3G%2BPDqofoBb5NE6get8bBb9zZ%2FA4SH24X%2BXfSmc%3D&reserved=0.

timotheeguerin commented 5 years ago

Should look like this image

SharmainePopov commented 5 years ago

Ok! No errors but no graphs either 😊 ☹

Sharmaine Popov Cloud Solution Architect |Microsoft Canada Inc. | sharmaine.popov@microsoft.commailto:sharmaine.popov@microsoft.com | 416-799-0807

From: Timothee Guerin notifications@github.com Sent: Tuesday, June 11, 2019 12:36 PM To: Azure/batch-insights batch-insights@noreply.github.com Cc: Sharmaine Popov Sharmaine.Popov@microsoft.com; Author author@noreply.github.com Subject: Re: [Azure/batch-insights] No Metrics Showing (#39)

In Batch Explorer after you update the start task you get a notification that lets you reboot all the nodes at once too.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FAzure%2Fbatch-insights%2Fissues%2F39%3Femail_source%3Dnotifications%26email_token%3DAIIVNWWRPZZ5J5NXADBJG7LPZ7H77A5CNFSM4HW2AXFKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXNXXTA%23issuecomment-500923340&data=02%7C01%7CSharmaine.Popov%40microsoft.com%7Cb0413dac1d684ba622b108d6ee8aec93%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636958677780173497&sdata=pcpL1sjpEmALqrhM04wWxT4LSEkUA1l7xxGfrE3x1g0%3D&reserved=0, or mute the threadhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAIIVNWSGL3PAZ2NXXFVTCTDPZ7H77ANCNFSM4HW2AXFA&data=02%7C01%7CSharmaine.Popov%40microsoft.com%7Cb0413dac1d684ba622b108d6ee8aec93%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636958677780173497&sdata=uHML3G%2BPDqofoBb5NE6get8bBb9zZ%2FA4SH24X%2BXfSmc%3D&reserved=0.

SharmainePopov commented 5 years ago

Do you have a working demo? Would like to show my client something in about 15 minutes 😊

Sharmaine Popov Cloud Solution Architect |Microsoft Canada Inc. | sharmaine.popov@microsoft.commailto:sharmaine.popov@microsoft.com | 416-799-0807

From: Timothee Guerin notifications@github.com Sent: Tuesday, June 11, 2019 12:40 PM To: Azure/batch-insights batch-insights@noreply.github.com Cc: Sharmaine Popov Sharmaine.Popov@microsoft.com; Author author@noreply.github.com Subject: Re: [Azure/batch-insights] No Metrics Showing (#39)

Should look like this [image]https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fuser-images.githubusercontent.com%2F1031227%2F59290196-db777700-8c2c-11e9-8c2b-30a30bc0fc81.png&data=02%7C01%7CSharmaine.Popov%40microsoft.com%7Ce921f855256943c3203708d6ee8b73b9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636958680058844890&sdata=7gozNRvp8XsOCoB5iWblyrrFAKtObYiqP5zfO9qvjSQ%3D&reserved=0

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FAzure%2Fbatch-insights%2Fissues%2F39%3Femail_source%3Dnotifications%26email_token%3DAIIVNWVOGYYKJQVZJZTFDVDPZ7IOFA5CNFSM4HW2AXFKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXNYDUI%23issuecomment-500924881&data=02%7C01%7CSharmaine.Popov%40microsoft.com%7Ce921f855256943c3203708d6ee8b73b9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636958680058844890&sdata=YO555DFmtTbKZnDCAEZSJ4tTuRz4EiVx1YFQ8Fhmc%2BI%3D&reserved=0, or mute the threadhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAIIVNWUHNRVK67JQE3ORQYTPZ7IOFANCNFSM4HW2AXFA&data=02%7C01%7CSharmaine.Popov%40microsoft.com%7Ce921f855256943c3203708d6ee8b73b9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636958680058854897&sdata=%2FVHrgNkyRM7JkduppkKzwGLBJ%2FpeMfvbA7L7Yra62Lw%3D&reserved=0.

SharmainePopov commented 5 years ago

Ok! Got graphs. Is there a way to persist this data or they can just look at the graphs??

Sharmaine Popov Cloud Solution Architect |Microsoft Canada Inc. | sharmaine.popov@microsoft.commailto:sharmaine.popov@microsoft.com | 416-799-0807

From: Timothee Guerin notifications@github.com Sent: Tuesday, June 11, 2019 12:40 PM To: Azure/batch-insights batch-insights@noreply.github.com Cc: Sharmaine Popov Sharmaine.Popov@microsoft.com; Author author@noreply.github.com Subject: Re: [Azure/batch-insights] No Metrics Showing (#39)

Should look like this [image]https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fuser-images.githubusercontent.com%2F1031227%2F59290196-db777700-8c2c-11e9-8c2b-30a30bc0fc81.png&data=02%7C01%7CSharmaine.Popov%40microsoft.com%7Ce921f855256943c3203708d6ee8b73b9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636958680058844890&sdata=7gozNRvp8XsOCoB5iWblyrrFAKtObYiqP5zfO9qvjSQ%3D&reserved=0

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FAzure%2Fbatch-insights%2Fissues%2F39%3Femail_source%3Dnotifications%26email_token%3DAIIVNWVOGYYKJQVZJZTFDVDPZ7IOFA5CNFSM4HW2AXFKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXNYDUI%23issuecomment-500924881&data=02%7C01%7CSharmaine.Popov%40microsoft.com%7Ce921f855256943c3203708d6ee8b73b9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636958680058844890&sdata=YO555DFmtTbKZnDCAEZSJ4tTuRz4EiVx1YFQ8Fhmc%2BI%3D&reserved=0, or mute the threadhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAIIVNWUHNRVK67JQE3ORQYTPZ7IOFANCNFSM4HW2AXFA&data=02%7C01%7CSharmaine.Popov%40microsoft.com%7Ce921f855256943c3203708d6ee8b73b9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636958680058854897&sdata=%2FVHrgNkyRM7JkduppkKzwGLBJ%2FpeMfvbA7L7Yra62Lw%3D&reserved=0.

timotheeguerin commented 5 years ago

App insights has a delay in the metrics ~5min so would make sense nothing is there yet. I can give you read permission to an account which has test pool but graphs will look like that image

timotheeguerin commented 5 years ago

All this data is in application insights which persist for 90 days I believe. You can query it yourself and get with specific parameters. See this doc for it https://github.com/Azure/batch-insights/wiki/Query-application-insights

SharmainePopov commented 5 years ago

Wait have graphs – but no CPU? Also do you know if code can be instrumented wit App Insights to get the task run data? I went through this article, with the accompanying code but never got anything in App Insights. https://docs.microsoft.com/en-us/azure/batch/monitor-application-insights

Sharmaine Popov Cloud Solution Architect |Microsoft Canada Inc. | sharmaine.popov@microsoft.commailto:sharmaine.popov@microsoft.com | 416-799-0807

From: Timothee Guerin notifications@github.com Sent: Tuesday, June 11, 2019 12:40 PM To: Azure/batch-insights batch-insights@noreply.github.com Cc: Sharmaine Popov Sharmaine.Popov@microsoft.com; Author author@noreply.github.com Subject: Re: [Azure/batch-insights] No Metrics Showing (#39)

Should look like this [image]https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fuser-images.githubusercontent.com%2F1031227%2F59290196-db777700-8c2c-11e9-8c2b-30a30bc0fc81.png&data=02%7C01%7CSharmaine.Popov%40microsoft.com%7Ce921f855256943c3203708d6ee8b73b9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636958680058844890&sdata=7gozNRvp8XsOCoB5iWblyrrFAKtObYiqP5zfO9qvjSQ%3D&reserved=0

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FAzure%2Fbatch-insights%2Fissues%2F39%3Femail_source%3Dnotifications%26email_token%3DAIIVNWVOGYYKJQVZJZTFDVDPZ7IOFA5CNFSM4HW2AXFKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXNYDUI%23issuecomment-500924881&data=02%7C01%7CSharmaine.Popov%40microsoft.com%7Ce921f855256943c3203708d6ee8b73b9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636958680058844890&sdata=YO555DFmtTbKZnDCAEZSJ4tTuRz4EiVx1YFQ8Fhmc%2BI%3D&reserved=0, or mute the threadhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAIIVNWUHNRVK67JQE3ORQYTPZ7IOFANCNFSM4HW2AXFA&data=02%7C01%7CSharmaine.Popov%40microsoft.com%7Ce921f855256943c3203708d6ee8b73b9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636958680058854897&sdata=%2FVHrgNkyRM7JkduppkKzwGLBJ%2FpeMfvbA7L7Yra62Lw%3D&reserved=0.

timotheeguerin commented 5 years ago

CPU should be here, what do you see? For the task specific metrics. this tool unfortunately doesn't provide this functionality. For this you'd need to have something run for every task.

SharmainePopov commented 5 years ago

[cid:image001.png@01D52054.E7BD6B30]

Sharmaine Popov Cloud Solution Architect |Microsoft Canada Inc. | sharmaine.popov@microsoft.commailto:sharmaine.popov@microsoft.com | 416-799-0807

From: Timothee Guerin notifications@github.com Sent: Tuesday, June 11, 2019 12:53 PM To: Azure/batch-insights batch-insights@noreply.github.com Cc: Sharmaine Popov Sharmaine.Popov@microsoft.com; Author author@noreply.github.com Subject: Re: [Azure/batch-insights] No Metrics Showing (#39)

CPU should be here, what do you see? For the task specific metrics. this tool unfortunately doesn't provide this functionality. For this you'd need to have something run for every task.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FAzure%2Fbatch-insights%2Fissues%2F39%3Femail_source%3Dnotifications%26email_token%3DAIIVNWXG3MES4O45TTZIKCDPZ7KAPA5CNFSM4HW2AXFKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXNZQHI%23issuecomment-500930589&data=02%7C01%7CSharmaine.Popov%40microsoft.com%7C0a2b149d92aa4fc2027a08d6ee8d53aa%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636958688096609800&sdata=LYGAgWWFXXKg1jq1MBmdP23tcUU5XuRhWEbwI%2Bzmb%2F0%3D&reserved=0, or mute the threadhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAIIVNWTCFUCEQSOZVOTOUCLPZ7KAPANCNFSM4HW2AXFA&data=02%7C01%7CSharmaine.Popov%40microsoft.com%7C0a2b149d92aa4fc2027a08d6ee8d53aa%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636958688096609800&sdata=Qq%2FChMFu%2BEGogqaFNDI5aT0iymjSJ%2FfcIu%2BCkxSgtkc%3D&reserved=0.

timotheeguerin commented 5 years ago

The image doesn't show up on github or the email. Could you try uploading via github?

SharmainePopov commented 5 years ago

missingcpu

Hi, missing CPU

timotheeguerin commented 5 years ago

Hhm and no errors in nodestats.log or nodestats.err.log? Which OS is the pool running on?

SharmainePopov commented 5 years ago

Nope, I’ll send the out log later. Can we get per node stats in log analytics

Sharmaine Popov Cloud Solution Architect |Microsoft Canada Inc. | sharmaine.popov@microsoft.commailto:sharmaine.popov@microsoft.com | 416-799-0807

From: Timothee Guerin notifications@github.com Sent: Tuesday, June 11, 2019 1:02 PM To: Azure/batch-insights batch-insights@noreply.github.com Cc: Sharmaine Popov Sharmaine.Popov@microsoft.com; Author author@noreply.github.com Subject: Re: [Azure/batch-insights] No Metrics Showing (#39)

Hhm and no errors in nodestats.log or nodestats.err.log? Which OS is the pool running on?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FAzure%2Fbatch-insights%2Fissues%2F39%3Femail_source%3Dnotifications%26email_token%3DAIIVNWT5HWATAAZKPQJ2YHTPZ7LCDA5CNFSM4HW2AXFKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXN2NQA%23issuecomment-500934336&data=02%7C01%7CSharmaine.Popov%40microsoft.com%7Ce6f9347c8b58468a530008d6ee8e93fa%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636958693469122823&sdata=SRBbkd26vocGAE%2F8V3Ai4%2B3AE7B8974a9xnE%2B2nwEPE%3D&reserved=0, or mute the threadhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAIIVNWWOZQ4R5N6GPGWNOWTPZ7LCDANCNFSM4HW2AXFA&data=02%7C01%7CSharmaine.Popov%40microsoft.com%7Ce6f9347c8b58468a530008d6ee8e93fa%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636958693469122823&sdata=TaTVoonvDoJlDjM06qp2TCjsuwSDMJrFHmGuJJfu05E%3D&reserved=0.

timotheeguerin commented 5 years ago

Yes, you can query using the roleInstance property

https://github.com/Azure/batch-insights/wiki/Query-application-insights#basic-list-of-all-metrics-for-a-node

SharmainePopov commented 5 years ago

Hi Thanks for all the help, I think the client can use this. Do you know what the data collection interval is for the performance counters?

Sharmaine Popov Cloud Solution Architect |Microsoft Canada Inc. | sharmaine.popov@microsoft.commailto:sharmaine.popov@microsoft.com | 416-799-0807

From: Timothee Guerin notifications@github.com Sent: Tuesday, June 11, 2019 1:07 PM To: Azure/batch-insights batch-insights@noreply.github.com Cc: Sharmaine Popov Sharmaine.Popov@microsoft.com; Author author@noreply.github.com Subject: Re: [Azure/batch-insights] No Metrics Showing (#39)

Yes, you can query using the roleInstance property

https://github.com/Azure/batch-insights/wiki/Query-application-insights#basic-list-of-all-metrics-for-a-nodehttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FAzure%2Fbatch-insights%2Fwiki%2FQuery-application-insights%23basic-list-of-all-metrics-for-a-node&data=02%7C01%7CSharmaine.Popov%40microsoft.com%7Ca62d858e05714b4cc38808d6ee8f329e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636958696135982526&sdata=5WMt4%2BVpoT3svmcMpgALA4OuhbujPxhwGJ3nN0XwZ%2B4%3D&reserved=0

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FAzure%2Fbatch-insights%2Fissues%2F39%3Femail_source%3Dnotifications%26email_token%3DAIIVNWT7YKLIOO2DWFN4LTDPZ7LSXA5CNFSM4HW2AXFKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXN22PQ%23issuecomment-500935998&data=02%7C01%7CSharmaine.Popov%40microsoft.com%7Ca62d858e05714b4cc38808d6ee8f329e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636958696135982526&sdata=ow2Vx7T4QCiRMgS1JSS1yBqOxt%2BRGNXfoSZBk%2Beh1yo%3D&reserved=0, or mute the threadhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAIIVNWVPPZ6GF33RIJAM3SLPZ7LSXANCNFSM4HW2AXFA&data=02%7C01%7CSharmaine.Popov%40microsoft.com%7Ca62d858e05714b4cc38808d6ee8f329e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636958696135982526&sdata=A3v%2Fy3iUd5JBn0TtvW%2BMgT5%2Bg8zERy5iKS46raiCQf8%3D&reserved=0.

timotheeguerin commented 5 years ago

You can check the config options here https://github.com/Azure/batch-insights/blob/master/docs/configuration.md

One is the local aggregation interval. By default it collect metrics every 5s, aggregate and uploads every minute(which is the minimum app insights will aggregate on. That way it reduce the amount of data uploaded to app insights and so the cost)

SharmainePopov commented 5 years ago

Thanks again. Any chance we can make the aggregation interval less than 1 minute - or otherwise get at the raw data? My client is a research lab and they are going to be doing a bench mark paper, and they were hoping they could get the data the need via a cloud service rather than trying instrument the OS.

Sharmaine Popov Cloud Solution Architect |Microsoft Canada Inc. | sharmaine.popov@microsoft.commailto:sharmaine.popov@microsoft.com | 416-799-0807

From: Timothee Guerin notifications@github.com Sent: Tuesday, June 11, 2019 7:15 PM To: Azure/batch-insights batch-insights@noreply.github.com Cc: Sharmaine Popov Sharmaine.Popov@microsoft.com; Author author@noreply.github.com Subject: Re: [Azure/batch-insights] No Metrics Showing (#39)

You can check the config options here https://github.com/Azure/batch-insights/blob/master/docs/configuration.mdhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FAzure%2Fbatch-insights%2Fblob%2Fmaster%2Fdocs%2Fconfiguration.md&data=02%7C01%7CSharmaine.Popov%40microsoft.com%7C1939d660dc94463053a608d6eec2a0d7%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636958917051368341&sdata=qRwRlxbN%2Bkx6mW2Iz9OBBFEiFP310boFehtaxRpUVLQ%3D&reserved=0

One is the local aggregation interval. By default it collect metrics every 5s, aggregate and uploads every minute(which is the minimum app insights will aggregate on. That way it reduce the amount of data uploaded to app insights and so the cost)

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FAzure%2Fbatch-insights%2Fissues%2F39%3Femail_source%3Dnotifications%26email_token%3DAIIVNWURRBYOVL7ZMRUSODTP2AWXJA5CNFSM4HW2AXFKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXOYVVA%23issuecomment-501058260&data=02%7C01%7CSharmaine.Popov%40microsoft.com%7C1939d660dc94463053a608d6eec2a0d7%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636958917051368341&sdata=9u27buOetT7Zw%2B7oJ9LPoqDikjR5cbBKUVaK6VDTlss%3D&reserved=0, or mute the threadhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAIIVNWXYRUFDXUO52HYC77TP2AWXJANCNFSM4HW2AXFA&data=02%7C01%7CSharmaine.Popov%40microsoft.com%7C1939d660dc94463053a608d6eec2a0d7%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636958917051524596&sdata=94C3PsGaDeVrGI5nwTMo28NDcNlHGmj2FQleeC2Rytc%3D&reserved=0.

timotheeguerin commented 5 years ago

Unfortunately even if batch-insights upload every 5s, I believe application insights will only aggregate the data in a minimum of a minute interval. You have to be carful with the cost of application insights which can become quite high with large pools if you upload too much data. You could always take this project and change it to upload to another system(like graphana) and get the raw data this way

SharmainePopov commented 5 years ago

Ok – where would I start. Is this developer stuff?

Sharmaine Popov Cloud Solution Architect |Microsoft Canada Inc. | sharmaine.popov@microsoft.commailto:sharmaine.popov@microsoft.com | 416-799-0807

From: Timothee Guerin notifications@github.com Sent: Wednesday, June 12, 2019 11:11 AM To: Azure/batch-insights batch-insights@noreply.github.com Cc: Sharmaine Popov Sharmaine.Popov@microsoft.com; Author author@noreply.github.com Subject: Re: [Azure/batch-insights] No Metrics Showing (#39)

Unfortunately even if batch-insights upload every 5s, I believe application insights will only aggregate the data in a minimum of a minute interval. You have to be carful with the cost of application insights which can become quite high with large pools if you upload too much data. You could always take this project and change it to upload to another system(like graphana) and get the raw data this way

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FAzure%2Fbatch-insights%2Fissues%2F39%3Femail_source%3Dnotifications%26email_token%3DAIIVNWRI6TGKB5QFC5PYZKTP2EGWXA5CNFSM4HW2AXFKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXQYEHQ%23issuecomment-501318174&data=02%7C01%7CSharmaine.Popov%40microsoft.com%7Cd45f58d18eae4787dd1808d6ef481f63%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636959490375791671&sdata=5NIxf2%2BHN%2F%2BRVHIq9FaKgnVPDpa8N2LoQqqhfWz0TYU%3D&reserved=0, or mute the threadhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAIIVNWT6GP43QGGGDTO63X3P2EGWXANCNFSM4HW2AXFA&data=02%7C01%7CSharmaine.Popov%40microsoft.com%7Cd45f58d18eae4787dd1808d6ef481f63%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636959490375791671&sdata=Ey%2FBB7LqqL29oeA9wd93yTZiKGh43QFSjpOBAcbUEcg%3D&reserved=0.

timotheeguerin commented 5 years ago

Yes you would need to fork the project and implement the data egress for another system than application insights.