[Proposal] Add support for describing different types of usage of a single product.

AWS-ZachErdman commented 8 months ago

Type

Type of issue: Dimension Normalized? No

Description

Cloud providers often charge customers for multiple types of operations for the same service (e.g., data transfer and data storage fees for an Object Storage service). It’s important for cloud customers to be able to clearly understand when they are being charged for each type of service operation.

Currently, the FOCUS spec provides the SKU ID column for cloud providers which can be used to differentiate between different types of operations for the same Service. However, a SKU is not human-interpretable, so customers will need additional information in order to understand a given line item charge. Instead of having each cloud provider create their own column to describe the type of usage in a line item (linked with the SKU), the FOCUS specification should standardize on a Column that provides the human-interpretable language describing the type of usage.

We suggest adding a column named “UsageType” to address this use case. A draft proposal of the column is provided below:

Column Proposal

Explanation  The UsageType column describes what type of operation is being charged for in the given line item. For a given time interval, a single cloud resource may be charged for multiple types of usage. This column enables cloud customers to understand what usage is being charged for in a format that is more human-interpretable than the SKU.  

The Usage Type column has the following requirements: 1. UsageType SHOULD be present in the billing data 2. Line items with the same SkuId MUST NOT have different usage type values

Sample data for Amazon S3 USE1-APS5-AWS-In-Bytes, Requests-Tier1

Description The type of usage being charged for in a given line item.

Column Constraints Column type : dimension Column required: FALSE Allows nulls: TRUE Data type: string Value format: not specified

Definition of done

[ ] Rationalize vendor-neutral, cross-cloud naming
[ ] Complete spec template and include naming (code name, display name), constraints, guidelines, compatibility with major providers etc.

goeln commented 8 months ago

+1 to this!

udam-f2 commented 8 months ago

Definitely a very important need. This has been discussed as a part of the SKU. @AWS-ZachErdman thoughts on if this can be a fast follow or if you consider this a P0?

cnharris10 commented 8 months ago

flanakin commented 8 months ago

Absolutely! This is what #315 is all about 🙂 We discussed this leading up to the preview and just weren't able to lock it down. We were in agreement that it needs to be multiple columns so we're not merging multiple things into a single column, so we won't have "UsageType" as a column, but instead break apart the different constructs that are part of UsageType in AWS.

AWS-ZachErdman commented 8 months ago

Definitely a very important need. This has been discussed as a part of the SKU. @AWS-ZachErdman thoughts on if this can be a fast follow or if you consider this a P0?

I would consider this P0 since we would likely need to add this as a column ourselves to the AWS FOCUS. We could end up putting switching costs on customers for AWS (and potentially other clouds as well) that we may be able to avoid now if we know we'll be adding it.

AWS-ZachErdman commented 8 months ago

Absolutely! This is what #315 is all about 🙂 We discussed this leading up to the preview and just weren't able to lock it down. We were in agreement that it needs to be multiple columns so we're not merging multiple things into a single column, so we won't have "UsageType" as a column, but instead break apart the different constructs that are part of UsageType in AWS.

@flanakin Would this be another good opportunity for a single column with a key-value construct since each cloud provider provides different ways of breaking down the SKU?

AWS-ZachErdman commented 8 months ago

I'd be interested in starting a task force on this if we haven't yet, or joining the existing one!

udam-f2 commented 8 months ago

That's awesome... Let us discuss this at the maintainer meeting tomorrow and revert.

If not in v1.0, the option would be for AWS to provide its usage type column with the provider namespacing. I'm worried about whether we'll be able to fit all these changes into the v1.0 schedule but happy to pressure test with maintainers and get an answer.

< personal view> we tried to fit all SKU details into a fixed scheme for almost a month. There's no mapping that works across all providers and products. So yes, absolutely for a JSON field where the provider provides the appropriate details. In various discussions, we've called this SKUDetails. Now UsageType specifically though seems like it might be a prominent/common enough thing across all providers that maybe it deserves it's own column so people can access easily without JSON parsing? The same pattern may apply for ResourceDetails and CommitmentDetails to fit various other properties). </personal view>

Happy to have you lead it if we scope to v1.0 or you can drive it right after.

gparker-at-sf commented 5 months ago

Not sure if I can assign myself; reporting in here.

jpradocueva commented 5 months ago

Karl will add the notes from the subtask discussion to the issue.

jpradocueva commented 5 months ago

Action Items:

[ ] TS2-#339 Graham to share the comparison of old and new Azure taxonomy layers with the group.
[ ] TS2-#339 Update taxonomy definitions to clearly distinguish between service and usage perspectives.
[ ] TS2-#339Review the mapping of resource types to service names and service categories for consistency.
[ ] TS2-#339: Michael, Irena: To draft the initial structure of the SKU description column and its attributes.
[ ] TS2-#339: Andrew: To review and provide feedback on the drafted structure.
[ ] TS2-#339: Michael and Andrew: To collaborate on a document detailing the attributes of SKU price IDs across different cloud providers.
[ ] TS2-#339: Team: To map existing attributes in the focus data model to those used by various cloud providers and determine any gaps.

kk09v commented 4 months ago

Adding my comment from the 20240626 meeting into this thread for discussion.

Previously, I had suggested that SKU Description is 1:1 with SKU ID, but upon further reflection, I realized that there are SKU Description might be part of a composite set of columns that would be 1:1 (with a few edge cases) to SKU ID. For instance, if a provider has a different SKU ID for each region that the same service is operating in, I personally would want the SKU Description to describe the thing being purchased, regardless of which region it's running in.

ijurica commented 4 months ago

I propose we close this issue since it will be addressed as part of issue #495

AWS-ZachErdman commented 4 months ago

Agree with Irena - maybe we can close and link to 495 for history

jpradocueva commented 4 months ago

The group agreed to close this issue on TF-2, July 17 call.

FinOps-Open-Cost-and-Usage-Spec / FOCUS_Spec