Closed dhoppla closed 6 months ago
andy-maier
commented
1 year ago @dhoppla The exporter is already today supposed to show such changes automatically.
Can you please produce a log file from the zhmc prometheus exporter that shows the JMS interactions with the HMC when these changes are done, and attach it?
You can create that using the options --log-comp all=debug --log hmc.log
dhoppla
commented
1 year ago hmc.log Hi, in this case we have made following changes to the MSU: LPAR MXA0 old 98 MSU new 99MSU LPAR ZZ3E old 778 MSU new 777 MSU
In the zHMC there are still the old MSU values:
zhmc_partition_cp_current_processing_weight{cpc="XX3L",hmc="XX2ZHMC",partition="XX3LMXA0",pod="mypod"} 98.0 zhmc_partition_cp_current_processing_weight{cpc="XX3L",hmc="XX2ZHMC",partition="XX3LZZ3E",pod="mypod"} 778.0
dhoppla
commented
10 months ago Hi, do they need further documentation ?
andy-maier
commented
9 months ago Hello @dhoppla - the hmc.log file does not contain much useful information for this issue, but I can try to reproduce it locally.
andy-maier
commented
9 months ago @dhoppla I started analyzing the issue.
For the zhmc_partition_defined_capacity_msu_per_hour metric, the reason it is not changing is that the corresponding property defined-capacity on the "Logical Partition" object does not support property change notifications. The zhmc exporter subscribes for property change notifications for all of its metrics that are resource-based. These are the metrics in the metric definition file that have "type: resource". So the exporter relies on property change notifications being sent from the HMC to the exporter, whenever the value changes. Apparently we defined at least this one metric without double checking that it actually supports property change notfications. I will need to check this for all other resource-based metrics, and then figure out whether implementing a circumvention makes sense (the only circumvention I know is to fetch the LPAR properties when Prometheus asks for updated metrics, which will make the response time to Prometheus larger. Stay tuned.
For the zhmc_partition_current_central_memory_mib metric: This metric supports property change notifications, but I did not find a way to change the central storage on an LPAR that is operating. How did you change the central storage? Did you change it in the image activation profile and then re-activated the LPAR? Or was this a change driven by z/OS where a storage area was varied on?
For the zhmc_partition_cp_current_processing_weight metric: That metric is not representing the defined capacity, but the relative processing weight. Its value has the unit of a weight relative to all other shared LPARs. The defined capacity has an MSU/h unit. If the two values are the same then that's a coincidence, IMO. I have to admit that I'm not a classic mode expert though, so you can possibly educate me on why you expected the current weight to change.
dhoppla
commented
8 months ago Hi, we change the central storage in the image activation profile and re-activated the LPAR.
We get many orders from our customers to change the current MSU settings during operation. So it would be nice to see the current values.
andy-maier
commented
6 months ago Sorry for the delay. I am resuming the work on this issue now.
andy-maier
commented
6 months ago I have investigated the resource properties that are shown as metrics by the zhmc prometheus exporter regarding whether they support property change notifications, and the following properties can change, but do not support change notifications:
LPAR properties (classic mode):
CPC properties:
For both CPC and LPAR, the retrieval operations support specifying a list of properties, so we can fetch just these properties, which should be faster than retrieving everything. I'll make measurements to see whether that is affordable, timewise.
andy-maier
commented
6 months ago The measurements are encouraging, but not encouraging enough to fetch these properties synchronously when Prometheus requests metrics from the exporter. I guess we need to build an async mechanism for that.
andy-maier
commented
6 months ago @dhoppla I have started implementing support for fetching these properties in the background. See PR #469
andy-maier
commented
6 months ago @dhoppla PR https://github.com/zhmcclient/zhmc-prometheus-exporter/pull/469 is ready for use.
You can try it out by installing it from the branch of that PR:
pip install zhmc-prometheus-exporter@git+https://github.com/zhmcclient/zhmc-prometheus-exporter.git@andy/fetch-propertiesPlease make sure that you use the new metric definition file: https://github.com/zhmcclient/zhmc-prometheus-exporter/blob/andy/fetch-properties/examples/metrics.yaml
Describe the bug We have changed the MSU value from 12 to 13 for an LPAR on HMC, but unfortunately the new value is not displayed in the zHMC. In another LPAR we have enlarged the central memory from 8192 to 9216, this value is also not displayed in the zHMC. In both cases, only the old value is displayed.
zhmc_partition_current_central_memory_mib{cpc="XXXX",hmc="XXXXHMC",partition="XXXXCF36",pod="mypod"} 8192.0 zhmc_partition_defined_capacity_msu_per_hour{cpc="XXXX",hmc="XXXXHMC",partition="XXXXXXXX",pod="mypod"} 12.0
Expected behavior It would be nice that these changes could be automatically displayed in the zHMC.
To Reproduce We changed the MSU value on HMC from 12 to 13 and also the central memory vom 8192MB to 9216 MB
Environment information
zhmc_prometheus_exporter --version:Command output <-- Relevant parts of the command output. If possible, with '-vv'. -->
Log file <-- If possible, attach a log file generated with '--log-comp all=debug --log exporter.log'. -->