[viewer api] quota and consumption for hdd storage types must be returned

[antonkovalenko]

As a follow-up for #3430

1. Info about unused storage types must not be returned in tenantInfo/storageUsage response.
2. Currently database cold-visits with HDD storage without storage database quotas doesn't have HDD info in tenantInfo/storageUsage response. But must have info about quotas and consumption of attached HDD storage

[StekPerepolnen]

https://github.com/ydb-platform/ydb/pull/5069

The expected result looks like this.

"StorageUsage": [
     {
        "Type": "SSD",
        "Size": "12345", 
        "Limit": "12345", 
        "SoftQuota": "12345", 
        "HardQuota": "12345",
    },
    {
        "Type": "HDD",
        "Size": "12345", 
        "Limit": "12345", 
        "SoftQuota": "12345", 
        "HardQuota": "12345",
    },
   {
        "Type": "None",
        "Size": "12345", 
        "Limit": "12345", 
        "SoftQuota": "12345", 
        "HardQuota": "12345",
    }
]

I'm going to remove QuotaUsage field like before and combine info from it with StorageUsage instead I would suggest that Size and Limit will go to the main Info card, and quotas will go to the details.

The pool type is specified in the config. It's a String that don't have to be like SSD or HDD. So I added the block None in case the name is not parsed correctly - this usage and quotas will go to None.

[StekPerepolnen]

I would like all the logic for calculating values for the Size and Limit to remain within the handler. Currently, HardQuota is always placed there. @artemmufazalov Could you clarify in which cases other values might be used?

[artemmufazalov]

@StekPerepolnen

I'm going to remove QuotaUsage field like before and combine info from it with StorageUsage instead I would suggest that Size and Limit will go to the main Info card, and quotas will go to the details.

StorageUsage and QuotaUsage not only have different quotas, but have different current size. We need both sizes and both limits for extended info. Don't merge them, please

@artemmufazalov Could you clarify in which cases other values might be used?

Recently, we agreed to use soft quota, so our diagnostics will match with data that ydb provides to monitoring services (soft quota as limit). So we need soft quota as limit as well.

I suggested to add both quotas in other ticket, so there will be flexibility to change values, when needed. Currently, hard quota is not needed at all. At the same time, we discussed displaying all data as table in extended info, where there will be not only consumption for every disk type, but also all quotas (both soft and hard)

[StekPerepolnen]

StorageUsage and QuotaUsage not only have different quotas, but have different current size. We need both sizes and both limits for extended info. Don't merge them, please

@artemmufazalov There is an issue with the sizes and limits not being separated by storage type as they were before. I would like to keep only one field, StorageUsage, for different storage types.

The sizes and limits that can't be separated by storage type are still accessible through other fields. They are not divided by storage type:

Metrics.Storage - Blob storage size StorageAllocatedSize - Tablet storage size StorageAllocatedLimit - Tablet storage limit

so the final response could look like this

"Metrics": {
    "Storage": "12345"
},
"StorageAllocatedSize": "12345",
"StorageAllocatedLimit": "12345",
"StorageUsage": [
     {
        "Type": "SSD",
        "Size": "12345", 
        "Limit": "12345", 
        "SoftQuota": "12345", 
        "HardQuota": "12345",
    },
    {
        "Type": "HDD",
        "Size": "12345", 
        "Limit": "12345", 
        "SoftQuota": "12345", 
        "HardQuota": "12345",
    },
   {
        "Type": "None",
        "Size": "12345", 
        "Limit": "12345", 
        "SoftQuota": "12345", 
        "HardQuota": "12345",
    }
]

[artemmufazalov]

StorageUsage and QuotaUsage not only have different quotas, but have different current size

@StekPerepolnen, here I mean blob and tablet storage, we need both. We use tablet storage if quotas are set and blob storage size and limits otherwise

[StekPerepolnen]

have to be TabletStorage and DatabaseStorage