[Security Solution] Implement a Zod transformation from snake_case to camelCase and vice versa

jpdjere commented 3 months ago

Epics: https://github.com/elastic/security-team/issues/1974 (internal), https://github.com/elastic/kibana/issues/174168 Needed for: https://github.com/elastic/kibana/issues/180124

Summary

We have API schemas where we use snake_casing for object properties, and we have internal BE-side schemas for rule parameters and saved object attributes where we use camelCasing for object properties. Because of that, we manually transform objects between the two schemas, and every time we add a new rule field, we need to write two schemas for it and two transformations between them

https://github.com/elastic/kibana/blob/7f0701d352fcfe8635ba0357c50b5722ae8a2226/x-pack/plugins/security_solution/common/api/detection_engine/model/rule_schema/common_attributes.gen.ts#L449-L461

https://github.com/elastic/kibana/blob/7f0701d352fcfe8635ba0357c50b5722ae8a2226/x-pack/plugins/security_solution/server/lib/detection_engine/rule_management/utils/utils.ts#L346-L366

To simplify writing and maintaining schemas, we should try to implement a reusable Zod transformations from snake_case to camelCase, something like that:

export type SnakeCasedObject = z.infer<typeof SnakeCasedObject>;
export const SnakeCasedObject = z.object({
  nested_object: z.object({
    some_property: z.string(),
  }),
});

export type CamelCasedObject = z.infer<typeof CamelCasedObject>;
export const CamelCasedObject = SnakeCasedObject.transform(toCamelCase);

// Should yield:

/*
type CamelCasedObject = {
  nestedObject: {
    someProperty: string;
  };
};
*/

Notes

Some ideas how to do that can be found in a GitHub issue: https://github.com/colinhacks/zod/issues/486#issuecomment-1567296747.

This transformation should:

Support recursion: it should be possible to apply it to deeply nested objects.
Support arrays of primitives and arrays of deeply nested objects.
Support type intersection and union, including discriminated union. This will be needed for defining types for the rule_source field:
- https://github.com/elastic/kibana/issues/180122
- https://github.com/elastic/kibana/issues/180124

During implementation, please evaluate if it would be helpful to have a vice versa toSnakeCase Zod transformation, in addition to toCamelCase.

Note that toCamelCase would be in fact a schema and type transformation that would make it easier to define internal schemas that correspond to API schemas. It wouldn't be a data transformation (something we have in rule_converters.ts). To make it easier to write and maintain functions like convertAlertSuppressionToCamel and convertAlertSuppressionToSnake we might additionally want to write reusable functions for transforming data. Functions from lodash could be useful for that: _.camelCase, _.snakeCase, _.mapKeys.

Background

Context from the RFC:

https://github.com/elastic/kibana/blob/07706fd8e5b423e7e9f4eae096fae24d93d64c2c/x-pack/plugins/security_solution/docs/rfcs/detection_response/prebuilt_rules_customization.md?plain=1#L313-L330

elasticmachine commented 3 months ago

Pinging @elastic/security-detections-response (Team:Detections and Resp)

elasticmachine commented 3 months ago

Pinging @elastic/security-solution (Team: SecuritySolution)

elasticmachine commented 3 months ago

Pinging @elastic/security-detection-rule-management (Team:Detection Rule Management)

xcrzx commented 2 months ago

The approach proposed in the ticket didn't work as expected, see https://github.com/elastic/kibana/pull/184004 for more detail. So I am reopening it. We need to investigate other options for automatic case conversation in schemas

elastic / kibana