Transitional Field Statuses: Allow fields to be marked 'introduced' and 'removing' in addition to 'required' and 'optional'

[clord]

Problem

Modifying required fields in our API can lead to compatibility issues between clients and servers, especially when fields are added, removed, or have their required/optional status changed. The key challenges are:

1. When adding a new required field, clients using older versions of the API may not include the new required field in their requests to the updated server, resulting in validation errors and failed requests.
2. When removing a required field and a client is updated before the server, it may stop sending a field that the server still expects as required, causing processing errors on the server side.
3. Promoting optional fields to required, or demoting required fields to optional, can create inconsistencies and unexpected behaviours if not managed carefully.

Making all fields optional is not a feasible solution because:

• It increases the complexity of runtime validation.
• It allows for invalid or meaningless combinations of data, potentially leading to logical errors in the application.

Proposal: Transitional Field Statuses

Implement transitional statuses for fields to manage their lifecycle without breaking existing client/servers:

becoming-required — introducing a new field that will eventually become required

• Writers (data producers) must include this field in their output.
• Readers (data consumers) should handle cases where this field is absent.

becoming-optional — planning to deprecate and eventually remove an existing required field

• Writers must continue to include this field in their output.
• Readers should prepare to handle cases where this field might be missing in the future.

This approach allows for a gradual transition.

• When adding new fields, mark the new field as becoming-required to ensure writers start including it while readers remain compatible with data that may not contain it. Once all writers are updated, we can then mark the field as required, so that clients can begin assuming it's always present.
• When removing fields, mark the field as becoming-optional to signal that it will eventually be removed, allowing readers to adjust without immediate impact on writers. Once all readers are updated to no longer expect the field, we can remove the field, and writers can stop emitting the field.

Reader/Writer vs Client/Server

Please note the distinction between client/server and reader/writer. Both clients and servers write to structs, and read from structs. In this PR we draw the distinction instead between readers and writers.

[matryer]

@clord thanks for this - the 'required' thing in Grafana Incident is handled via the comment metadata mechanism in Oto. The templates check that field and render appropriate validation output. So I think this would be best solved in the templates themselves, and you don't need to change Oto to do it. Oto doesn't have any specific support for 'required' fields.

[clord]

Template changes will be required, but this request is for something between optional and required. When introducing a new field, or removing it, we want the validation of the producer to be required, but the validation of the consumer to be optional. This way, we start producing it asap, but we don't have to validate it yet on the consumer. imo, we want to break the two 'sides' of required up up so template can know which case we are in.