Update/Re-generate OpenMetadataTypes archive documentation

[ilovechai]

Is there an existing issue for this?

• [X] I have searched the existing issues

Current Behavior

The UML diagram https://egeria-project.org/types/5/0512-Derived-Schema-Elements/ below describes that a CalculatedValue classification can be added to a SchemaElement.

Which should mean a RelationalTable whose superType is > SchemaAttribute > SchemaElement should be able to add the CalculatedValue classifiation

https://egeria-project.org/types/5/0505-Schema-Attributes/?h=schema#schemaattributetype-relationship

This is link to the supported OpenMetadataTypes https://raw.githubusercontent.com/odpi/egeria/master/content-packs/OpenMetadataTypes.json which contains the CalculatedValue ClassificationDef that supports SchemaType instead, needs to be updated to SchemaElement

{
"class": "ClassificationDef",
"headerVersion": 1,
"guid": "4814bec8-482d-463d-8376-160b0358e139",
"name": "CalculatedValue",
"status": "ACTIVE_TYPEDEF",
"version": 1,
"versionName": "1.0",
"category": "CLASSIFICATION_DEF",
"description": "A field within a schema that is calculated via the formula and query targets rather than stored.",
"origin": "bce3b0a0-662a-4f87-b8dc-844078a11a6e",
"createdBy": "ODPi Egeria",
"createTime": 1588261366992,
"validInstanceStatusList": [
"ACTIVE",
"DELETED"
],
"initialStatus": "ACTIVE",
"propertiesDefinition": [
{
"headerVersion": 1,
"attributeName": "formula",
"attributeType": {
"class": "PrimitiveDef",
"headerVersion": 1,
"version": 1,
"versionName": "1.0",
"category": "PRIMITIVE",
"guid": "b34a64b9-554a-42b1-8f8a-7d5c2339f9c4",
"name": "string",
"primitiveDefCategory": "OM_PRIMITIVE_TYPE_STRING"
},
"attributeStatus": "ACTIVE_ATTRIBUTE",
"attributeDescription": "Expression to create the value.",
"valuesMinCount": 0,
"valuesMaxCount": 1,
"indexable": true,
"attributeCardinality": "AT_MOST_ONE",
"unique": false
}
],
"validEntityDefs": [
{
"headerVersion": 1,
"guid": "5bd4a3e7-d22d-4a3d-a115-066ee8e0754f",
"name": "SchemaType",
"status": "ACTIVE_TYPEDEF"
}
],
"propagatable": false
}

Expected Behavior

This is link to the supported OpenMetadataTypes https://raw.githubusercontent.com/odpi/egeria/master/content-packs/OpenMetadataTypes.json which contains the CalculatedValue ClassificationDef that supports SchemaType instead, needs to updated to be changed to SchemaElement

Steps To Reproduce

No response

Environment

- Egeria: 3.10
- OS:
- Java: 11
- Browser (for UI issues):
- Additional connectors and integration:

Any Further Information?

No response

[mandy-chessell]

Open Metadata Types are versioned.

Version 1 of CalculatedValue did connect it to SchemaType. However this was updated to connecting to SchemaElement in Version 2.

The way types are updated is through TypeDefPatches. Version 1 of the type appears in the archive as a ClassificationDef. Version 2 of this type is present in the archive as a TypeDefPatch.

When the archive is loaded by the server, the typedefs (ie the version 1 of all of the types) are loaded first. Then the TypeDefPatches are loaded to make the changes.

I can see both versions of CalculatedValue in the archive and version 2 is correctly set up in the server when it starts.

I am not sure there is anything to change since the right values are in the archive and the latest type version is running in the server.

[mandy-chessell]

Here is the link to the structure of the archive - https://egeria-project.org/features/metadata-archiving/overview/#inside-an-open-metadata-archive

[ilovechai]

@mandy-chessell thank you, I now see the version 2 of CalculatedValue in the link https://egeria-project.org/features/metadata-archiving/overview/#inside-an-open-metadata-archive

When the archive is loaded by the server, the typedefs (ie the version 1 of all of the types) are loaded first. Then the TypeDefPatches are loaded to make the changes.

We are on egeria 3.10, at runtime we get this error

2022-10-18 08:44:09 ERROR - mapClassifications error: OCFCheckedExceptionBase{reportedHTTPCode=400, reportingClassName='org.odpi.openmetadata.repositoryservices.localrepository.repositorycontentmanager.OMRSRepositoryContentHelper', reportingActionDescription='getSkeletonClassification', reportedErrorMessage='OMRS-REPOSITORY-400-006 Open metadata repository CalculatedValue is unable to assign a classification of type CalculatedValue to an entity of type RelationalTable as the classification type is not valid for this type of entity', reportedErrorMessageId='OMRS-REPOSITORY-400-006', reportedErrorMessageParameters=[CalculatedValue, RelationalTable], reportedSystemAction='The system is not able to classify an entity since the ClassificationDef for the classification does not list the entity type, or one of its super-types.', reportedUserAction='Update the ClassificationDef to include the entity's type and rerun the request. Alternatively use a different classification.', reportedCaughtException=null, reportedCaughtExceptionClassName='null', relatedProperties=null}
org.odpi.openmetadata.repositoryservices.ffdc.exception.TypeErrorException: OMRS-REPOSITORY-400-006 Open metadata repository CalculatedValue is unable to assign a classification of type CalculatedValue to an entity of type RelationalTable as the classification type is not valid for this type of entity
    at org.odpi.openmetadata.repositoryservices.localrepository.repositorycontentmanager.OMRSRepositoryContentHelper.getSkeletonClassification(OMRSRepositoryContentHelper.java:962) ~[repository-services-implementation-3.10.jar!/:?]
    at org.odpi.openmetadata.repositoryservices.localrepository.repositorycontentmanager.OMRSRepositoryContentHelper.getSkeletonClassification(OMRSRepositoryContentHelper.java:860) ~[repository-services-implementation-3.10.jar!/:?]
    at com.ibm.iis.openmetadata.adapters.igc.mapper.source.AssetMapper.mapClassifications(AssetMapper.java:627) [igc-catalog-connector-11.7.1.1-SNAPSHOT.jar!/:?]

We are using the default OpenMetadataTypesArchive(at egeria 3.10 the archive version is 2.7) provided by egeria 3.10. Do we need to patch the repository content manager or should this automatically be done by egeria? When the TypeDefPatches are loaded to make the changes, is the repository content manager updated by default?

[mandy-chessell]

Yes, the repository content manager is updated with the typedef patches. This is how the service runs normally in the OMAGServerPlatform.

I would suggest that you re-run OpenMetadataTypesArchiveUtility to generate OpenMetadataTypesArchive.json for 3.10.

This is the first time that anyone has said that they use the OpenMetadataTypesArchive.json which is why we update it infrequently. The OMAGServerPlatform uses the Java call OpenMetadataTypesArchive.getOpenMetadataArchive() to get the latest types ... which may be a better solution for you

[ilovechai]

@mandy-chessell Thank you! We use the OpenMetadataTypesArchive.json just for reference purpose and no where in code. Patching the igc(internal) metadata store worked for me. Shouldn't the internal metadata store be patched with OpenMetadataTypes automatically by the egeria repository content manager?

[mandy-chessell]

The internal type store is located within the egeria repository content manager.