Partition.list(full_properties=True) performance improvement by using bulk operation

[andy-maier]

The Partition.list() method with full_properties=True performs a "List Partitions of CPC" operation, followed by a loop of "Get Partition Properties" operations.

The loop can be avoided by performing a "Get Inventory" operation on resource "partition", which returns all partitions of all CPCs and their child properties such as NICs.

[andy-maier]

PR #331 has been provided a long time ago to address this. I recently resumed the work on it up to the point where the performance can be measured using a new example script (in PR #1163).

The performance results are not very encouraging for this improvement:

On the HMC of T224 etc, which has:

• CPC M224M with 19 partitions (4 not stopped)
• CPC T224 with 29 partitions (19 not stopped)
• CPC A224 with 0 partitions
• A total of 326 inventory items for resource "partition" (= the 48 partitions plus their child elements such as NICs, HBAs, VFs)

For T224, Partition.list() has these results:

• with full_properties=False: 0.14 - 0.16 sec, 4 properties per partition
• with full_properties=True based on get partition loop (i.e. without the PR): 5.45 - 5.57 sec, 4 properties per partition
• with full_properties=True based on get inventory (i.e. with the PR): 7.50 - 7.88 sec, 87.6 properties per partition

The worse elapsed time is probably caused by the many additional inventory items that are returned (326 items for 48 partitions).

[andy-maier]

There is also the possibility of using a bulk operation with the 48 get partition properties operations, to see how that performs.

[andy-maier]

PR #1244 implements use of the bulk operation (aggregation service) in case full_properties has been set.

The performance result determined by the examples/list_partitions_partial_and_full.py script is:

Using HMC at 9.114.87.7 (2.16.0) and CPC M224M:

Listing partitions with full_properties=False without the PR: Duration: 0.19 s Number of partitions: 19 Number of non-stopped partitions: 5 Average number of properties per partition: 4.0

Listing partitions with full_properties=True without the PR: Duration: 5.47 s Number of partitions: 19 Number of non-stopped partitions: 5 Average number of properties per partition: 85.1

Listing partitions with full_properties=True with the PR: Duration: 1.25 s Number of partitions: 19 Number of non-stopped partitions: 5 Average number of properties per partition: 85.1

So the PR reduces the time for 19 partitions from 5.47 s to 1.25 s.

This is very promising and will be used for all resource objects (after the centralization of the list() method in PR #1241 has been merged).