Add User Defined Fields / Dimensions

[dr00b]

Is your feature request related to a problem? Please describe. In many analytical contexts, users want to bring fields outside the standard data model for slicing and dicing downstream:

• These are typically low cardinality fields, which have some meaning to the local business context. For instance, NPI's may map to "pods", essentially groups of NPI's that coordinate care within a health system. So, every claim might be mapped to pod1, pod2, pod3, with the rest tossed and unknown pod.
• Another low cardinality example is something like "is deceased". Perhaps there's an inferred version based on lack of utilization which should be kept separate from a higher quality data source. Both are relevant filters / slicers depending on the use case.
• You may also have higher cardinality fields, For instance, the concatenation of a business key at the claim level. You want to pass this concatenated value downstream to make it easy to join back to source data sources. If you don't pass the concatenated value downstream, users of the analytical data mart, end up having to concatenate fields at query time to reconstruct the join. This is more error prone than just handling what the queriers want up front.

Describe the solution you'd like Interacting with various vendors, I've seen this done a few ways. The best pattern, I think, is like so:

• Create "user defined dimensions", precompute the cube: these are fields by which the data will be grouped automatically as part of the mart. When the mart runs, precompute analytical products by these dimensions (PMPM, quality measures, etc.). Probably best to keep them all as string, and trim them...
• Create "user defined fields": these are the high cardinality fields, or fields that didn't justify the compute to for slicers.
• Support rename back to original user local name: user should be able to specify their local name for the "user defined field" and have it provided back to them when processing completes. So... MyGarbargeLocalName -> udf_1 (for local processing) -> MyGarbageLocalName (in analytical products / tables). As long as there is no collision with standard field names, this is fine. I'm a bit of a DBT noob, not sure if this would require macros to be this dynamic.

Describe alternatives you've considered Won't name names, but I have seen this pattern in various professional services companies (tax accounting, finance, pop health analytics, etc.).

[thutuva]

Hi @dr00b - Thank you for submitting this idea. I appreciate you suggestions on how we can accommodate custom fields into our data model. The Tuva data model governance team will discuss this and we'll get back to you!