dell / PyU4V

PyU4V is a Python module that simplifies interaction with the Unisphere for PowerMax REST API. It wraps REST calls with simple APIs that abstracts the HTTP request and response handling.
Apache License 2.0
41 stars 37 forks source link

No Allocated details in get_srp() function #170

Closed 212850a closed 1 year ago

212850a commented 1 year ago

HI,

Solutions Enabler (SE) reports about SRP with Allocated details:

symcfg -sid SID list -srp -detail -tb
                           STORAGE RESOURCE POOLS
Symmetrix ID  : SID
                                                 C A P A C I T Y
-------------------------------- --- -------------------------------------------
                                 Flg     Usable  Allocated       Free Subscribed
Name                             DR        (TB)       (TB)       (TB)       (TB)
-------------------------------- --- ---------- ---------- ---------- ----------
SRP_1                            FX      293.40     241.16      52.23     676.31
                                     ---------- ---------- ---------- ----------
Total                                    293.40     241.16      52.23     676.31

provisioning.get_srp('SRP_1') function for some reason returns usable_total_tb and usable_used_tb, but usable_used_tb is less than Allocated from SE output, so it's not the same. performance.get_array_stats(metric='ALL',start_time,end_time) returns both AllocatedCapacityTB and Used_Usable_CapTB.

Question - is it possible to get AllocatedCapacityTB with a help of get_srp() or I have to use performance.get_array_stats for that as there is no other way how to get it?

rawstorage commented 1 year ago

What array model are you using, the figues returned will be different e.g. 8500/2500 will return
"fba_srp_capacity": { "provisioned": { "provisioned_tb": 24.72, "effective_capacity_tb": 229.78, "provisioned_percent": 11 },

Here you have provisioned TB value which is what is provisioned to user applications, details on all the values are in the unisphere help will break out by array model and microcode.d

212850a commented 1 year ago

We use AFA 450F and PMAX2000 with 5978.711.711 and they have the same output for get_srp(). There is no AllocatedCapacityTB in output, only usable_used_tb which does not include System Used and Temp Used which Unisphere is reporting about. AllocatedCapacityTB = User Used (usable_used_tb) + System Used + Temp Used

image001

So as result from get_srp() we can't state what is whole AllocatedCapacityTB as only usable_used_tb (User Used) is visible there.

rawstorage commented 1 year ago

Yes the numbers are different on V3 you will have numbers like the following "srp_capacity": { "subscribed_allocated_tb": 0.53, >>I believe this is the metric you need, Subscribed Allocated Capacity represents how much of the logical capacity of the SRP is allocated. "subscribed_total_tb": 77, "snapshot_modified_tb": 0, "snapshot_total_tb": 241.93, "usable_used_tb": 1.49, "usable_total_tb": 48.9, "effective_used_capacity_percent": 1

212850a commented 1 year ago

"subscribed_allocated_tb": 0.53, >>I believe this is the metric you need, Subscribed Allocated Capacity represents how much of the logical capacity of the SRP is allocated.

Not really, subscribed_allocated_tb is how much capacity presented to host(s), but we want to know how much really used. usable_used_tb gives not whole picture about it.

If you login to Unisphere and check Usable Capacity for array as on attached screenshot above you'll see that usable_used_tb is User Used. But where Unisphere gets System Used and Temp Used data from? AllocatedCapacityTB = User Used (usable_used_tb) + System Used + Temp Used

rawstorage commented 1 year ago

AFIK, Temp used should not be counted towards allocated capacity, this is space from SRP temporarily used and will be returned to the allocated pool. The numbers we have available in PyU4V are those that are reported by the API and Unipshere, TEMP was never exposed and it causes confusion I agree. I posted a blog on the numbers some time ago.

https://rawstorage.wordpress.com/2019/09/18/where-does-unisphere-get-its-powermax-efficiency-numbers-and-how-can-i-get-them-from-cli-rest-and-unisphere/

To add based on discussions I've had in the past on Temp . Temp space is just that – temp. Some of it is ABC data (Activity Based Compression data not currently reduced), There is no impact to the data reduction commitment, since temp should not be considered. It is being borrowed while the array is not full. It is not consumed space. Which is why the data reduction numbers use the efficiency reporting and not the SRP (temp) detail reporting.

Hope this helps.

212850a commented 1 year ago

The numbers we have available in PyU4V are those that are reported by the API and Unipshere, TEMP was never exposed and it causes confusion I agree.

So from where Unisphere gets "Temp Used" metric if it's not exposed via U4V? It's visible on attached above screenshot.

To add based on discussions I've had in the past on Temp . Temp space is just that – temp. Some of it is ABC data (Activity Based Compression data not currently reduced), There is no impact to the data reduction commitment, since temp should not be considered. It is being borrowed while the array is not full. It is not consumed space.

I agree that Temp can be ignored when array is not really used, but when array is utilised up to 70% and Temp Used is more than 10% of Usable Capacity it starts to play significant role when you need to decide how much capacity you still can present to hosts. Real example:

symcfg -sid 000111111111 list -tb -demand -v

Symmetrix ID                : 000111111111
Array Usage
  User Subscribed (TB)      :  724.86
    Allocation (%)          :  61
    Non Shared (TB)         : 444.56
    Shared     (TB)         :   0.00
  System Subscribed (TB)    :   1.21
  Snapshot Capacity (TB)    :   23.47
    Modified (%)            :  67
    Non Shared (TB)         :  14.83
    Shared (TB)             :   0.82
  Usable Capacity (TB)      : 293.40
    Used (%)                :  70
    Used Capacity (TB)      : 205.34
      User Used (TB)        : 204.13
      System Used (TB)      :   1.21
    Temp Used (TB)          :  30.07
  Encapsulated (TB)         :   0.00
Meta Data Usage
  System (%)                :  78
  Replication (%)           :   6
  Frontend (%)              :  36
  Backend (%)               :  67

So from where Unisphere gets "Temp Used" metric? We just want to collect it either to take right decision about left capacity for new provisionings.

rawstorage commented 1 year ago

Hi, It can appear that way but the Temp capacity is still available to you for provisioning operations, this was a confusing metric and is no longer presented. The remaining capacity for user allocations is still 30% however a lot of customers only ever provision arrays up to a certain point saving space for other operations and snapshot capacity growth. I suggest a conversation with you account SE, the metrics available are presented by the API and PyU4V, any decision you make on your capacity allocation model should be possible based on the available metrics without Temp. Temp space will always be available for user data if it is required by the system so should be disregarded in any calculation.

rawstorage commented 1 year ago

closing issue as this is with API and not PyU4V specifically,