Issues with setting data collection intervals

[mmichalo]

Hello, I am having issue with finding proper setting to setup intervals for picking up data via HMC API rest. Data collection is setup and intervals for 5 min. Json files are showing that interval is set to 30 sec

Where this can be changed? Also its picking up data for last 2h, is this possible to change it as well?

[nigelargriffiths]

Hi,

The HMC is collecting the data at the rate it wants - it is not decided by user. The data is also used by the HMC performance GUI.

The performance stats cache on the HMC only has the last 2 hours worth. If we collect the stats with over lapping time frames (and that is normal for nextract) we get back exactly the same numbers covering the overlap time. In the case of nextract and InfluxDB that does no harm - InfluxDB records only one set of numbers for each specific time when the values are the same. So it does no harm done - except repeatedly sending the same stats.

I am not clear if you want to collect more or less stats?

Nextract is coded assuming this simple approach will get the job done. And allow for the OS running nextract can be rebooted or updated. Provided it is back up and running in under 2 hours.

You could change the start and end times but then you will have to add that feature yourself to the Python It introduces a world of pain computing date and times plus recording some how what was the last data/time collected and saved correctly.

You will have to add to the data requests two extra fields in the Python For example the get request strings: /rest/api/pcm/ManagedSystem/{uuid}/RawMetrics/ShortTermMonitor?StartTS={StartTS}&EndTS={EndTS} To get you started look at the documentation: https://public.dhe.ibm.com/systems/power/docs/hw/p9/p9ehl.pdf

[cid:020665c9-fbed-4d38-b674-fa519b9ac1bb]

Good luck.

Cheers, Nigel Griffiths IBM retired, Advanced Technology, Power Systems, AIX & Linux Performance @mr_nmon Performance tools developer Open Source from: https://nmon.sourceforge.io/pmwiki.php and https://github.com/nigelargriffiths?tab=repositories and just for fun http://mr-nmon.com/ Power & AIX Videos: @.*** IBM Blogs: https://www.ibm.com/support/pages/aixpert-blog-nigel-griffiths-mrnmon

---

From: mmichalo @.> Sent: 03 July 2024 12:52 To: nigelargriffiths/nextract @.> Cc: Subscribed @.***> Subject: [nigelargriffiths/nextract] Issues with setting data collection intervals (Issue #2)

Hello, I am having issue with finding proper setting to setup intervals for picking up data via HMC API rest. Data collection is setup and intervals for 5 min. Json files are showing that interval is set to 30 sec

Screenshot.2024-07-03.135144.png (view on web)https://github.com/nigelargriffiths/nextract/assets/100427168/8ec0e387-6e48-4d88-8cac-08afef6d8113

Where this can be changed? Also its picking up data for last 2h, is this possible to change it as well? Screenshot.2024-07-03.133913.png (view on web)https://github.com/nigelargriffiths/nextract/assets/100427168/6effd917-14d6-49a7-af03-383fd91c933e

— Reply to this email directly, view it on GitHubhttps://github.com/nigelargriffiths/nextract/issues/2, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ALPHMWPB33ZWSTJR4ISHUGLZKPQZPAVCNFSM6AAAAABKJNDUKSVHI2DSMVQWIX3LMV43ASLTON2WKOZSGM4DQNBTGM3TOOA. You are receiving this because you are subscribed to this thread.Message ID: @.***>

[mmichalo]

Hi Nigel, Thank you for quick response. My goal is:

• Change intervals for metrics which are grabbed from HMC (its 30 sec , which means that InfluxDB space will grow fast)
• Have possibility to pickup data from HMC with bigger window than 2 hours. Currently,for example if InfuxDB crash or I will lose connectivity on HMC for more than 2h I will have empty data for that period.

[nigelargriffiths]

Hi, Interval: The HMC stats gathering is off or on. The user does not get to decide the period. Generally, InfluxDB is excellent at efficiently storing the data. I suggest you have a few TB of space for InfluxDB to avoid the problem. A few TB does not cost a lot these days. You can use InfluxDB space management commands to reduce the data size or truncate (delete older records).

• Check the InfluxDB manuals for details.

Window: The HMC stats cache is for 2 hours. The user does not get to decide the windows size. This would require a change to the HMC software - you will need to request this change for IBM to consider it. The issues is how long should the window be. And can the HMC support the extra database size. Don't forget the HMC support (from my memory) 48 Servers and 1000 LPARs. Then there is the REST API changes to request a larger size. If this change gets a high priority it may appear in the HMC software for Power11 time frame. Perhaps, you need to monitor for those failures and alert yourself to enable quicker fix times.

I hope this helps. Cheers Nigel

[mmichalo]

Hello, Thank you for you answers. In case that limit for data collection per HMC is 48, I had to split frames data collection into two HMC. Then I run nextract plus scripts for both HMC and place data in the same InfluxDB.