[FEATURE] Add new table relationship metadata using DESC

[YANG-DB]

Is your feature request related to a problem? We have introduction the SimpleSchema that is based on the concept of a well structured index which is based on a schema

A schema is associated to an index using the mapping structure. This mapping structure is also composable using the composed_of template capabilities.

A schema is also capable of reflecting entities relationships - they are defined in a proprietary way of adding this information to the template's metadata

For example a log's entity relationship to the trace entity is described in the log's mapping metadata section

 "_meta": {
        "description": "Simple Schema For Observability",
        "catalog": "observability",
        "type": "logs",
        "correlations": [
          {
            "field": "spanId",
            "foreign-schema": "traces",
            "foreign-field": "spanId"
          },
          {
           "field": "traceId",
            "foreign-schema": "traces",
            "foreign-field": "traceId"
           }
          ]
        }

What solution would you like?

I would like that the SQL / PPL DESC command would reflect this metadata Information in a similar way to other existing relational databases reflect such relationship constraints

We will need to add a representation for the relationship concept in the interface Table either by extending the existing Map<String, ExprType> getFieldTypes() API or by adding a new API to represent the concept of relationship between tables

What alternatives have you considered? A clear and concise description of any alternative solutions or features you've considered.

Do you have any additional context?

• Schema
• Observability Schema
• Correlations
• Logs Correlations mapping metadata

[penghuo]

Questions.

1. Is the _meta field stored in the index mapping?
2. Does the concept of a foreign field correspond to the foreign key concept in relational databases? If so, does it require data integrity checks during insert operations? reference, https://www.postgresql.org/docs/current/tutorial-fk.html
3. What is the expected output of the DESC command? Can you provide an example from existing relational databases?
4. How can the returned data from the DESC command be utilized?

[YANG-DB]

1. yes
2. the concept is similar to some extent but differs in the sense that it only enforce metadata integrity constraints so that it is mostly used as a way for the SQL to infer correlation routes and expect the query to be able to execute since the data type match
3. the describe call will return the next response for the correlated field:
   • a new correlation column would be added
   • the value of this column would contain a list of the correlated remote schema.fieldName for example: [traces.traceId, logs.traceId]
     4. This information can be used to create a correlation query that is a template and be added to an integration of used by a dashboard in a similar manner the existing trace analytics uses traceGroupId to correlate services with traces