Azure / azure-rest-api-specs

The source for REST API specifications for Microsoft Azure.
MIT License
2.63k stars 5.05k forks source link

Microsoft.Consumption PriceSheets API is Paginated - Generated Config Says Otherwise #21139

Open zbmowrey opened 1 year ago

zbmowrey commented 1 year ago

Using the Azure SDK for Python -> ConsumptionManagementClient and the price_sheet.get_by_billing_period action, I would expect to get all pricing for the given account and billing period.

The SDK returns 1000 results and a next link to retrieve the remaining results. The root cause is that the SDK doesn't see the price sheets API endpoint as paginated, so it doesn't attempt to walk the data set:

See: https://github.com/Azure/azure-rest-api-specs/blob/main/specification/consumption/resource-manager/Microsoft.Consumption/stable/2021-10-01/consumption.json

The JSON objects at 1295 and 1358 should include the x-ms-pageable property, specifying the next_link property, which does exist in the response:

image

navba-MSFT commented 1 year ago

@zbmowrey Apologies for the late reply. Thanks for reaching out to us and reporting this issue. We are looking into this issue and we will provide an update.

navba-MSFT commented 1 year ago

@zbmowrey We are looking into this. In the meantime, could you please answer the below ?

zbmowrey commented 1 year ago

What other alternatives were explored since this is not working beyond 100 entries ?

I am currently pulling the first 1000 results using the SDK, then getting the next_link property from that result, and building a requests object which uses the auth header and next_link to walk through the remaining data set.

Considering there is a breaking change now in fixing this and introducing a new property, What alternatives will yoiu be planning to follow?

I can understand how changing or removing a property creates a breaking change, but adding a new property should not break existing workflows. I am currently using the above workaround, which has me manually doing what the SDK should do (and does on many other endpoints) in the first place.

What is the business impact because of this issue ? (Ex: Number of users affected, revenue loss, any production down situations)

The business impact is that the Microsoft Azure SDK is inaccurately reporting Pricing data to all users of the SDK who retrieve Pricing data. Our revenue loss is approximately 2 days of engineering time spent isolating, identifying, debugging, and working around this missing field. I can only assume that other clients may experience similar impact.

navba-MSFT commented 1 year ago

@zbmowrey Thanks for getting back. The details have been shared with the product owners and the request is currently under review.

HarveyLink commented 7 months ago

Hi @navba-MSFT , .NET SDK had a external customer fired same issue like this: https://github.com/Azure/azure-sdk-for-net/issues/35334. Regarding this operation: https://github.com/Azure/azure-rest-api-specs/blob/45f5b5a166c75a878d0f5404e74bd1855ff48894/specification/consumption/resource-manager/Microsoft.Consumption/stable/2023-11-01/consumption.json#L1305 This is a GET operation, return an object which contains PriceSheet list inside its properties. We may need to do some customization for this operation in the SDK. We would like to know how the service expects the customer to use this API. What’s the exact content if we call the next link