[Work_Item]: Augment support for correction handling

[ijurica]

1. Problem Statement *

Describe the problem, issue, use case, or opportunity that this work item addresses. Include practitioner quotes illustrating real examples a) of questions being asked by practitioners and b) value unlocked by answering these questions, if available.

• What is the problem?: Explain the context and why it needs resolution.
• Impact: Describe how the problem affects users, systems, or the project.

While some correction handling rules have been incorporated into individual columns' normative requirements - primarily within metrics and related unit dimensions - the FOCUS specification lacks a comprehensive, unified framework (i.e., attribute) for managing corrections across all columns. The absence of an overarching correction handling standard may lead to inconsistencies, oversights, and ambiguities in the specification, particularly in multi-dimensional and metric-based data contexts, ultimately leading to misinterpretation and the generation of non-conforming correction records, compromising data integrity, reconciliation, and overall data quality.

To address this, an overarching correction handling attribute should be introduced. Additionally, a review of existing correction handling rules within individual columns' normative requirements is essential to ensure consistency and alignment with this unified approach.

2. Objective *

State the objective of this work item. What outcome is expected?

• Success Criteria: Define how success will be measured (e.g. metrics and KPIs).

The goal of this work item is to formalize a consistent approach to correction handling across all columns within the FOCUS dataset. By providing clear guidelines for producing conforming correction records and managing corrections across complex cost records in multi-dimensional and metric-based data contexts, this work item aims to minimize misinterpretation by both producers and consumers, ultimately enhancing data quality, integrity, and reliability.

Expected Outcomes:

• Clear, unambiguous rules and expectations for correction records as a whole, along with a unified approach to specifying correction handling normative requirements for individual columns, are established.
• An overarching correction handling attribute is introduced.
• Correction-related normative requirements for individual columns (e.g., metrics and related unit dimensions) are consistent and aligned with the agreed unified approach and overarching correction-handling attribute.

Success Criteria:

Success can be measured by improvements in clarity, consistency, and accuracy regarding correction handling, as well as positive feedback from providers and practitioners on the ease of producing and validating conforming correction records.

3. Supporting Documentation *

Include links to supporting documents such as:

• Data Examples: [Link to data or relevant files; DO NOT share proprietary information]
• Related Use Cases or Discussion Documents: [Link to discussion]
• PRs or Other References: [Link to relevant references]

Data Examples:

TBD

Prerequisites:

• Issue #578
• Issue #594

Related Issues:

• Issue #557
• Issue #511

4. Proposed Solution / Approach

Outline any proposed solutions, approaches, or potential paths forward. Do not submit detailed solutions; please keep suggestions high-level.

• Initial Ideas: Describe potential solution paths, tools, or technologies.
• Considerations: Include any constraints, dependencies, or risks.
• Feasibility: Include any information that helps quantify feasibility, such as perceived level of effort to augment the spec, or existing fields in current data generator exports.
• Benchmarks: Are there established best practices for solving this problem available to practitioners today (e.g. mappings from existing CSP exports that are widely used)?

Initial Ideas:

• Define clear, unambiguous rules for the production, validation, and application of correction records.
• Establish a unified approach to correction handling normative requirements for individual columns.
• Introduce an overarching correction handling framework (attribute) that addresses all FOCUS columns under a holistic correction-handling guideline, aligned with previously defined rules.
• Revisit existing correction handling-related normative requirements for individual columns to ensure consistency and alignment with the overarching correction handling attribute and the agreed unified approach.

Considerations:

• There is a possibility that the results of this work item could introduce backward incompatibilities. Therefore, potential impacts will be carefully assessed at each stage of the process (e.g., defining rules, specifying the correction-handling attribute, and revisiting existing correction-related normative requirements for individual columns).
• Criteria for backward compatibility assessment are out of scope for this issue but must be prepared as a prerequisite.
• Normative requirements for existing FOCUS columns and attributes will be reviewed comprehensively in a separate issue.

Feasibility:

• Since this issue involves establishing rules and guidelines, introducing an overarching attribute, and addressing multiple columns, dividing the effort into multiple PRs will help manage the overall workload more effectively.

5. Epic or Theme Association

This section will be completed by the Maintainers.

• Epic: [Epic Name]
• Theme: [Theme Name, if applicable]

TBD

6. Stakeholders *

List the main stakeholders for this issue.

• Primary Stakeholder: [Name/Role]
• Other Involved Parties: [Names/Roles]

TBD

[jpradocueva]

Summary TF-2 call on Oct 16:

#556 [SPEC CHANGE]: Add Correction Handling Attribute and/or Appendix Key Discussion Items: The discussion centered around adding a correction handling attribute and ensuring corrections are correctly specified across all charge categories. Problem Identification: Corrections for metrics, costs, and unit prices are inconsistently handled. Divergent Views: Some participants suggested this is a minor issue, while others felt it needed more thorough specification. Final Agreement: Irena will create a work item for this topic using the work item template. Action Items: [TF-2-#556] Irena, @ijurica was assigned to backfill the issue or create a new work item following the standard template.

[jpradocueva]

Notes from the Maintainers' call on November 4:

Context: Current spec guidelines for cost corrections are insufficient, leading to discrepancies in how corrections are applied. This task aims to provide more comprehensive correction handling guidance to improve data consistency and accuracy. Level of Effort Required: Medium — Implementing improved correction handling is feasible but requires collaboration to define and document correction practices clearly.

[jpradocueva]

Summary Maintaiers's call on Nov 25

Context: This item aims to enhance the specification's ability to address and process corrections in billing data, ensuring accuracy in scenarios such as error adjustments or retroactive changes. Maintainers Assigned: Irena, probably Alex (Shawl will check with him) Task Force Assigned: Task Force 1 (TF1).