odpi / egeria

Egeria core
https://egeria-project.org
Apache License 2.0
807 stars 261 forks source link

Invalid AnchorGUID for comment #4680

Closed mandy-chessell closed 3 years ago

mandy-chessell commented 3 years ago

Noticed a curious exception coming from testing the labs:

Wed Feb 10 19:50:24 GMT 2021 cocoMDS5 Event OMRS-AUDIT-8006 Processing incoming event of type NewEntityEvent for instance b2c4f13b-7f84-44e7-a451-3d968e547f29 from: OMRSEventOriginator{metadataCollectionId='e89b978d-da68-4aec-8001-3bb46f32aee9', serverName='cocoMDS1', serverType='Metadata Server', organizationName='Coco Pharmaceuticals'}
Wed Feb 10 19:50:24 GMT 2021 cocoMDS2 Exception OMAS-GOVERNANCE-ENGINE-0007 The Governance Engine Open Metadata Access Service (OMAS) encounter an exception while processing event from Governance Engine OMAS of type processNewEntityEvent for instance b2c4f13b-7f84-44e7-a451-3d968e547f29
Wed Feb 10 19:50:24 GMT 2021 cocoMDS2 Exception OMAS-GOVERNANCE-ENGINE-0007 Supplementary information: log record id fbfa0fd7-2fe3-4995-a3fa-ee71aa6bd1de org.odpi.openmetadata.frameworks.connectors.ffdc.InvalidParameterException returned message of OMAG-REPOSITORY-HANDLER-404-007 The Referenceable entity with unique identifier <unknown> is not found for method processNewEntityEvent of access service Governance Engine OMAS in open metadata server cocoMDS2, error message was: OMRS-REPOSITORY-404-002 The entity identified with guid <unknown> passed on the getEntityDetail call is not known to the open metadata repository cocoMDSx and stacktrace of 
InvalidParameterException{parameterName='anchorGUID', reportedHTTPCode=404, reportingClassName='org.odpi.openmetadata.commonservices.repositoryhandler.RepositoryErrorHandler', reportingActionDescription='processNewEntityEvent', reportedErrorMessage='OMAG-REPOSITORY-HANDLER-404-007 The Referenceable entity with unique identifier <unknown> is not found for method processNewEntityEvent of access service Governance Engine OMAS in open metadata server cocoMDS2, error message was: OMRS-REPOSITORY-404-002 The entity identified with guid <unknown> passed on the getEntityDetail call is not known to the open metadata repository cocoMDSx', reportedErrorMessageId='OMAG-REPOSITORY-HANDLER-404-007', reportedErrorMessageParameters=[Referenceable, <unknown>, processNewEntityEvent, Governance Engine OMAS, cocoMDS2, OMRS-REPOSITORY-404-002 The entity identified with guid <unknown> passed on the getEntityDetail call is not known to the open metadata repository cocoMDSx], reportedSystemAction='The system is unable to update information associated with the entity because none of the connected open metadata repositories recognize the entity's unique identifier.', reportedUserAction='The unique identifier of the entity is supplied by the caller.  Verify that the caller's logic is correct, and that there are no errors being reported by the open metadata repository. Once all errors have been resolved, retry the request.', reportedCaughtException=OCFCheckedExceptionBase{reportedHTTPCode=404, reportingClassName='org.odpi.openmetadata.repositoryservices.clients.LocalRepositoryServicesClient', reportingActionDescription='getEntityDetail', reportedErrorMessage='OMRS-REPOSITORY-404-002 The entity identified with guid <unknown> passed on the getEntityDetail call is not known to the open metadata repository cocoMDSx', reportedErrorMessageId='OMRS-REPOSITORY-404-002', reportedErrorMessageParameters=[<unknown>, getEntityDetail, cocoMDSx], reportedSystemAction='The system is unable to retrieve the properties for the requested entity because the supplied guid is not recognized.', reportedUserAction='The guid is supplied by the caller to the server.  It may have a logic problem that has corrupted the guid, or the entity has been deleted since the guid was retrieved.', reportedCaughtException=null, reportedCaughtExceptionClassName='null', relatedProperties=null}, reportedCaughtExceptionClassName='org.odpi.openmetadata.repositoryservices.ffdc.exception.EntityNotKnownException', relatedProperties=null}
    at org.odpi.openmetadata.commonservices.repositoryhandler.RepositoryErrorHandler.handleUnknownEntity(RepositoryErrorHandler.java:551)
    at org.odpi.openmetadata.commonservices.repositoryhandler.RepositoryHandler.getEntityByGUID(RepositoryHandler.java:3185)
    at org.odpi.openmetadata.commonservices.generichandlers.OpenMetadataAPIGenericHandler.validateAnchorEntity(OpenMetadataAPIGenericHandler.java:2875)
    at org.odpi.openmetadata.commonservices.generichandlers.OpenMetadataAPIGenericHandler.getEntityFromRepository(OpenMetadataAPIGenericHandler.java:6795)
    at org.odpi.openmetadata.commonservices.generichandlers.OpenMetadataAPIGenericHandler.getBeanFromRepository(OpenMetadataAPIGenericHandler.java:6910)
    at org.odpi.openmetadata.commonservices.generichandlers.OpenMetadataAPIGenericHandler.getBeanFromRepository(OpenMetadataAPIGenericHandler.java:6874)
    at org.odpi.openmetadata.accessservices.governanceengine.handlers.MetadataElementHandler.getMetadataElementByGUID(MetadataElementHandler.java:122)
    at org.odpi.openmetadata.accessservices.governanceengine.outtopic.GovernanceEngineOMRSTopicListener.processWatchdogEvent(GovernanceEngineOMRSTopicListener.java:583)
    at org.odpi.openmetadata.accessservices.governanceengine.outtopic.GovernanceEngineOMRSTopicListener.processNewEntityEvent(GovernanceEngineOMRSTopicListener.java:714)
    at org.odpi.openmetadata.repositoryservices.connectors.omrstopic.OMRSTopicListenerBase.processInstanceEvent(OMRSTopicListenerBase.java:484)
    at org.odpi.openmetadata.repositoryservices.connectors.omrstopic.OMRSTopicListenerWrapper.processInstanceEvent(OMRSTopicListenerWrapper.java:146)
    at org.odpi.openmetadata.repositoryservices.connectors.omrstopic.OMRSTopicConnector.processOMRSEvent(OMRSTopicConnector.java:518)
    at org.odpi.openmetadata.repositoryservices.connectors.omrstopic.OMRSTopicConnector.lambda$processEvent$0(OMRSTopicConnector.java:455)
    at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
    at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
    at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
    at java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:291)
    at java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:731)
    at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
    at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
    at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
    at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
Caused by: OCFCheckedExceptionBase{reportedHTTPCode=404, reportingClassName='org.odpi.openmetadata.repositoryservices.clients.LocalRepositoryServicesClient', reportingActionDescription='getEntityDetail', reportedErrorMessage='OMRS-REPOSITORY-404-002 The entity identified with guid <unknown> passed on the getEntityDetail call is not known to the open metadata repository cocoMDSx', reportedErrorMessageId='OMRS-REPOSITORY-404-002', reportedErrorMessageParameters=[<unknown>, getEntityDetail, cocoMDSx], reportedSystemAction='The system is unable to retrieve the properties for the requested entity because the supplied guid is not recognized.', reportedUserAction='The guid is supplied by the caller to the server.  It may have a logic problem that has corrupted the guid, or the entity has been deleted since the guid was retrieved.', reportedCaughtException=null, reportedCaughtExceptionClassName='null', relatedProperties=null}
    at org.odpi.openmetadata.repositoryservices.clients.MetadataCollectionServicesClient.detectAndThrowEntityNotKnownException(MetadataCollectionServicesClient.java:5495)
    at org.odpi.openmetadata.repositoryservices.clients.MetadataCollectionServicesClient.getEntityDetail(MetadataCollectionServicesClient.java:1174)
    at org.odpi.openmetadata.adapters.repositoryservices.rest.repositoryconnector.OMRSRESTMetadataCollection.getEntityDetail(OMRSRESTMetadataCollection.java:893)
    at org.odpi.openmetadata.repositoryservices.enterprise.repositoryconnector.executors.GetEntityExecutor.issueRequestToRepository(GetEntityExecutor.java:107)
    at org.odpi.openmetadata.repositoryservices.enterprise.repositoryconnector.control.SequentialFederationControl.executeCommand(SequentialFederationControl.java:53)
    at org.odpi.openmetadata.repositoryservices.enterprise.repositoryconnector.EnterpriseOMRSMetadataCollection.getEntityDetail(EnterpriseOMRSMetadataCollection.java:983)
    at org.odpi.openmetadata.commonservices.repositoryhandler.RepositoryHandler.getEntityByGUID(RepositoryHandler.java:3177)
    ... 20 more

Nested in the initial message is:

OMAG-REPOSITORY-HANDLER-404-007 The Referenceable entity with unique identifier <unknown> is not found for method processNewEntityEvent of access service Governance Engine OMAS in open metadata server cocoMDS2, error message was: OMRS-REPOSITORY-404-002 The entity identified with guid <unknown> passed on the getEntityDetail call is not known to the open metadata repository cocoMDSx

This is occurring when the anchor entity is being validated.

On retrieving the entity from the repository it seems that the GUID in the Anchors classification is set to <unknown> and the repository services is quite right to throw an exception.

{
    "class": "EntityDetailResponse",
    "relatedHTTPCode": 200,
    "entity": {
        "class": "EntityDetail",
        "headerVersion": 1,
        "type": {
            "class": "InstanceType",
            "typeDefCategory": "ENTITY_DEF",
            "typeDefGUID": "1a226073-9c84-40e4-a422-fbddb9b84278",
            "typeDefName": "Comment",
            "typeDefVersion": 2,
            "typeDefDescription": "Descriptive feedback or discussion related to an item.",
            "typeDefSuperTypes": [
                {
                    "headerVersion": 1,
                    "guid": "a32316b8-dc8c-48c5-b12b-71c1b2a080bf",
                    "name": "Referenceable",
                    "status": "ACTIVE_TYPEDEF"
                },
                {
                    "headerVersion": 1,
                    "guid": "4e7761e8-3969-4627-8f40-bfe3cde85a1d",
                    "name": "OpenMetadataRoot",
                    "status": "ACTIVE_TYPEDEF"
                }
            ],
            "validInstanceProperties": [
                "qualifiedName",
                "additionalProperties",
                "anchorGUID",
                "commentType",
                "text",
                "type"
            ]
        },
        "instanceProvenanceType": "LOCAL_COHORT",
        "metadataCollectionId": "e89b978d-da68-4aec-8001-3bb46f32aee9",
        "metadataCollectionName": "Data Lake Catalog",
        "createdBy": "peterprofile",
        "createTime": "2021-02-10T19:50:24.489+00:00",
        "version": 1,
        "guid": "b2c4f13b-7f84-44e7-a451-3d968e547f29",
        "classifications": [
            {
                "class": "Classification",
                "headerVersion": 1,
                "type": {
                    "class": "InstanceType",
                    "typeDefCategory": "CLASSIFICATION_DEF",
                    "typeDefGUID": "aa44f302-2e43-4669-a1e7-edaae414fc6e",
                    "typeDefName": "Anchors",
                    "typeDefVersion": 2,
                    "typeDefDescription": "Identifies the anchor entities for an element that is part of a large composite object such as an asset.",
                    "validInstanceProperties": [
                        "anchorGUID"
                    ]
                },
                "instanceProvenanceType": "LOCAL_COHORT",
                "metadataCollectionId": "e89b978d-da68-4aec-8001-3bb46f32aee9",
                "metadataCollectionName": "Data Lake Catalog",
                "createdBy": "peterprofile",
                "createTime": "2021-02-10T19:50:24.460+00:00",
                "version": 1,
                "classificationOrigin": "ASSIGNED",
                "name": "Anchors",
                "properties": {
                    "class": "InstanceProperties",
                    "instanceProperties": {
                        "anchorGUID": {
                            "class": "PrimitivePropertyValue",
                            "instancePropertyCategory": "PRIMITIVE",
                            "typeGUID": "b34a64b9-554a-42b1-8f8a-7d5c2339f9c4",
                            "typeName": "string",
                            "primitiveDefCategory": "OM_PRIMITIVE_TYPE_STRING",
                            "primitiveValue": "<unknown>"
                        }
                    },
                    "propertyCount": 1,
                    "propertyNames": [
                        "anchorGUID"
                    ]
                },
                "status": "ACTIVE"
            }
        ],
        "properties": {
            "class": "InstanceProperties",
            "instanceProperties": {
                "commentType": {
                    "class": "EnumPropertyValue",
                    "instancePropertyCategory": "ENUM",
                    "ordinal": 0
                },
                "text": {
                    "class": "PrimitivePropertyValue",
                    "instancePropertyCategory": "PRIMITIVE",
                    "typeGUID": "b34a64b9-554a-42b1-8f8a-7d5c2339f9c4",
                    "typeName": "string",
                    "primitiveDefCategory": "OM_PRIMITIVE_TYPE_STRING",
                    "primitiveValue": "The hospital is still recruiting patients and so we are expecting a new version of this file at the end of week 4"
                },
                "qualifiedName": {
                    "class": "PrimitivePropertyValue",
                    "instancePropertyCategory": "PRIMITIVE",
                    "typeGUID": "b34a64b9-554a-42b1-8f8a-7d5c2339f9c4",
                    "typeName": "string",
                    "primitiveDefCategory": "OM_PRIMITIVE_TYPE_STRING",
                    "primitiveValue": "3c3026ac-7e7c-414c-bd9c-25de7507f097"
                }
            },
            "propertyCount": 3,
            "propertyNames": [
                "qualifiedName",
                "commentType",
                "text"
            ]
        },
        "status": "ACTIVE"
    }
}

So the important question is "Why is the AnchorGUID set to such a ridiculous value?"

mandy-chessell commented 3 years ago

Further investigation shows that this comment was added as part of the "Understanding an asset" lab in this code:

displayName = "Drop Foot Clinical Trial Patients"
description = "List of patients registered for the drop foot clinical trial."
fullPath    = "file://secured/research/clinical-trials/drop-foot/Patients.csv"
assetName   = "Patient Asset"

assetGUIDs = assetOwnerCreateCSVAsset(cocoMDS1Name, cocoMDS1PlatformName, cocoMDS1PlatformURL, petersUserId, displayName, description, fullPath)

assetGUID = getLastGUID(assetGUIDs)

print("Created " + assetName)

commentType = "STANDARD_COMMENT"
commentText = "The hospital is still recruiting patients and so we are expecting a new version of this file at the end of week 4"
isPublic    = True

commentGUID = addCommentToAsset(cocoMDS1Name,
                                cocoMDS1PlatformName,
                                cocoMDS1PlatformURL,
                                petersUserId,
                                assetGUID,
                                commentText,
                                commentType,
                                isPublic)

patientAssetOwner = "tanyatidie"
patientOwnerType  = "USER_ID"

addOwner(cocoMDS2Name, cocoMDS2PlatformName, cocoMDS2PlatformURL, erinsUserId, assetName, assetGUID, patientAssetOwner, patientOwnerType)
addZones(cocoMDS2Name, cocoMDS2PlatformName, cocoMDS2PlatformURL, erinsUserId, assetName, assetGUID, ["data-lake", "clinical-trials"])

print("Done.")

Which is this code

def addCommentToAsset(serverName, serverPlatformName, serverPlatformURL, userId, requestType, commentText, commentType, isPublic):
    assetConsumerURL = serverPlatformURL + '/servers/' + serverName + '/open-metadata/access-services/asset-consumer/users/' + userId 
    addCommentURL = assetConsumerURL + '/assets/' + requestType + '/comments'
    commentBody={
        "class" : "CommentProperties",
        "commentType" : commentType,
        "commentText" : commentText,
        "isPublic" : isPublic
    }
    response = issuePost(addCommentURL,commentBody)
    commentGUID = response.json().get('guid')
    if commentGUID:
        return commentGUID
    else:
        print ("No comment added")
        processErrorResponse(serverName, serverPlatformName, serverPlatformURL, response)

The assetGUID (which will become the anchorGUID) is being passed in as the requestType parameter which is being passed to the Asset Consumer OMAS as the assetGUID. So it is just a badly named parameter.

Next need to look at what is happening in the server.

mandy-chessell commented 3 years ago

Just run method again and both asset and comment are correctly set up ... this is the comment

{
    "class": "EntityDetailResponse",
    "relatedHTTPCode": 200,
    "entity": {
        "class": "EntityDetail",
        "headerVersion": 1,
        "type": {
            "class": "InstanceType",
            "typeDefCategory": "ENTITY_DEF",
            "typeDefGUID": "1a226073-9c84-40e4-a422-fbddb9b84278",
            "typeDefName": "Comment",
            "typeDefVersion": 2,
            "typeDefDescription": "Descriptive feedback or discussion related to an item.",
            "typeDefSuperTypes": [
                {
                    "headerVersion": 1,
                    "guid": "a32316b8-dc8c-48c5-b12b-71c1b2a080bf",
                    "name": "Referenceable",
                    "status": "ACTIVE_TYPEDEF"
                },
                {
                    "headerVersion": 1,
                    "guid": "4e7761e8-3969-4627-8f40-bfe3cde85a1d",
                    "name": "OpenMetadataRoot",
                    "status": "ACTIVE_TYPEDEF"
                }
            ],
            "validInstanceProperties": [
                "qualifiedName",
                "additionalProperties",
                "anchorGUID",
                "commentType",
                "text",
                "type"
            ]
        },
        "instanceProvenanceType": "LOCAL_COHORT",
        "metadataCollectionId": "e89b978d-da68-4aec-8001-3bb46f32aee9",
        "metadataCollectionName": "Data Lake Catalog",
        "createdBy": "peterprofile",
        "createTime": "2021-02-10T22:13:12.108+00:00",
        "version": 1,
        "guid": "3bf90b75-05f6-44f6-884b-4f96e5c2393f",
        "classifications": [
            {
                "class": "Classification",
                "headerVersion": 1,
                "type": {
                    "class": "InstanceType",
                    "typeDefCategory": "CLASSIFICATION_DEF",
                    "typeDefGUID": "aa44f302-2e43-4669-a1e7-edaae414fc6e",
                    "typeDefName": "Anchors",
                    "typeDefVersion": 2,
                    "typeDefDescription": "Identifies the anchor entities for an element that is part of a large composite object such as an asset.",
                    "validInstanceProperties": [
                        "anchorGUID"
                    ]
                },
                "instanceProvenanceType": "LOCAL_COHORT",
                "metadataCollectionId": "e89b978d-da68-4aec-8001-3bb46f32aee9",
                "metadataCollectionName": "Data Lake Catalog",
                "createdBy": "peterprofile",
                "createTime": "2021-02-10T22:09:58.253+00:00",
                "version": 1,
                "classificationOrigin": "ASSIGNED",
                "name": "Anchors",
                "properties": {
                    "class": "InstanceProperties",
                    "instanceProperties": {
                        "anchorGUID": {
                            "class": "PrimitivePropertyValue",
                            "instancePropertyCategory": "PRIMITIVE",
                            "typeGUID": "b34a64b9-554a-42b1-8f8a-7d5c2339f9c4",
                            "typeName": "string",
                            "primitiveDefCategory": "OM_PRIMITIVE_TYPE_STRING",
                            "primitiveValue": "abc58494-e027-4228-95f4-4bb907979803"
                        }
                    },
                    "propertyNames": [
                        "anchorGUID"
                    ],
                    "propertyCount": 1
                },
                "status": "ACTIVE"
            }
        ],
        "properties": {
            "class": "InstanceProperties",
            "instanceProperties": {
                "commentType": {
                    "class": "EnumPropertyValue",
                    "instancePropertyCategory": "ENUM",
                    "ordinal": 0
                },
                "text": {
                    "class": "PrimitivePropertyValue",
                    "instancePropertyCategory": "PRIMITIVE",
                    "typeGUID": "b34a64b9-554a-42b1-8f8a-7d5c2339f9c4",
                    "typeName": "string",
                    "primitiveDefCategory": "OM_PRIMITIVE_TYPE_STRING",
                    "primitiveValue": "The hospital is still recruiting patients and so we are expecting a new version of this file at the end of week 4"
                },
                "qualifiedName": {
                    "class": "PrimitivePropertyValue",
                    "instancePropertyCategory": "PRIMITIVE",
                    "typeGUID": "b34a64b9-554a-42b1-8f8a-7d5c2339f9c4",
                    "typeName": "string",
                    "primitiveDefCategory": "OM_PRIMITIVE_TYPE_STRING",
                    "primitiveValue": "481c068c-2854-4ced-9fec-01191c4541eb"
                }
            },
            "propertyNames": [
                "qualifiedName",
                "commentType",
                "text"
            ],
            "propertyCount": 3
        },
        "status": "ACTIVE"
    }
}
mandy-chessell commented 3 years ago

I am going to close this for the time being since I can not recreate it