Asset Catalog OMAS - exception on supportedTypes (impacts UI)

[planetf1]

The UI hits an exception on startup. The console logs (debug enabled) show:

2020-01-16 09:23:34.313 ERROR 37799 --- [nio-8443-exec-5] o.a.c.c.C.[.[.[/].[dispatcherServlet]    : Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is PropertyServerException{reportedHTTPCode=503, reportingClassName='org.odpi.openmetadata.accessservices.assetcatalog.AssetCatalog', reportingActionDescription='getSupportedTypes', errorMessage='OMAG-COMMON-503-001 A client-side exception was received from API call getSupportedTypes to OMAG Server cocoMDS4 at http://localhost:8081.  The error message was CLIENT-SIDE-REST-API-CONNECTOR-503-002 A client-side exception org.springframework.web.client.HttpServerErrorException$InternalServerError was received by method getSupportedTypes from API call http://localhost:8081/servers/{0}/open-metadata/access-services/asset-catalog/users/{1}/supportedTypes?type={2} to server cocoMDS4 on platform http://localhost:8081.  The error message was 500 : [{"timestamp":"2020-01-16T09:23:34.262+0000","status":500,"error":"Internal Server Error","message":"No message available","path":"/servers/cocoMDS4/open-metadata/access-services/asset-catalog/users/faithbroker/supportedTypes"}]', reportedSystemAction='The server has issued a call to the open metadata access service REST API in a remote server and has received an exception from the local client libraries.', reportedUserAction='Look for errors in the local server's console to understand and correct the source of the error.', reportedCaughtException=RESTServerException{reportedHTTPCode=503, reportingClassName='org.odpi.openmetadata.adapters.connectors.restclients.spring.SpringRESTClientConnector', reportingActionDescription='getSupportedTypes', errorMessage='CLIENT-SIDE-REST-API-CONNECTOR-503-002 A client-side exception org.springframework.web.client.HttpServerErrorException$InternalServerError was received by method getSupportedTypes from API call http://localhost:8081/servers/\{0\}/open-metadata/access-services/asset-catalog/users/\{1\}/supportedTypes\?type\=\{2\} to server cocoMDS4 on platform http://localhost:8081.  The error message was 500 : [{"timestamp":

Furthermore, an attempt to invoke this endpoint from the cli also fails with 500

✗ http GET http://localhost:8081/servers/cocoMDS4/open-metadata/access-services/asset-catalog/users/faithbroker/supportedTypes
HTTP/1.1 500
Connection: close
Content-Type: application/json
Date: Thu, 16 Jan 2020 09:38:00 GMT
Transfer-Encoding: chunked

{
    "error": "Internal Server Error",
    "message": "No message available",
    "path": "/servers/cocoMDS4/open-metadata/access-services/asset-catalog/users/faithbroker/supportedTypes",
    "status": 500,
    "timestamp": "2020-01-16T09:38:00.415+0000"
}

Originally noted in #2426

[planetf1]

Note that the UI does then allow searches to work correctly - at least of CSV assets @danielaotelea are you aware of this? Is it a problem for the 1.3 release?

[planetf1]

I note that in AssetCatalogSupportedTypes.java there were no annotations - without these spring returns back a 500 without invoking our code. With the annotations added, I still get a 500, which is caused by hitting an exception in AssetCatalogRESTService.java:getSupportedTypes when setting the response - that was down to an incomplete mapping

@danielaotelea - I assume we take no action immediately for 1.3 here. This looks like new code, and the impact on the UI isn't critical. It's work in progress you'll presumably fix in 1.4? Fair?

[bogdan-sava]

cannot reproduce do you have any logs ?

[danielaotelea]

Hi Nigel!

I'm not able to reproduce this issue.

[danielaotelea]

I have added the annotations, but there is no problem on our current environment without that annotations.

[planetf1]

Thanks...

I made the change to annotations locally - but it didn't help. I took your PR and built, but the same issue.

My environment is:

• MacOS
• master (up to date as of now)
• Java 11
• I launch 3 platforms & UI as per coco pharma environment
• I run through server configuration & server start notebooks, then asset management (this creates some assets and puts a variety of servers into a cohort)
• the error here occurs after login to the UI

Note that without the fix I made for #2432 (now merged to master, in queue for release 1.3) Asset Catalog OMAS was not working AT ALL - and hence nor would asset search in the UI. With that fix at least the UI works, and searches are successful, so it's just a warning error at login time.

I will attach the error, but my simpler test case was just using httpie (or use curl)

[planetf1]

The output I get when going to asset search in the UI is posted at https://gist.github.com/efcc3ae8b5588fec7e589c4d258d64cb

[bogdan-sava]

can you please provide also the configuration document for the server

[bogdan-sava]

can you please provide the response for getAllTypes

[danielaotelea]

Hi!

I have added a check for the active types. Do you have a active types available? There are some repositories in the cohort?

[bogdan-sava]

the only way we reproduce it is for the case there are no types available on server.

[bogdan-sava]

but once again we do not have any logs we have 500 response but no logs

[planetf1]

Apologies for the delay - calls, user issues & other 1.3 actions.

One method of reproducing is to run the coco pharma notebook environment - locally or in a container. Run the usual 'server configuration' and 'server start' notebooks, followed by 'asset-management/building a data catalog'. This reproduces the error every time.

The latest PR does stop the error. As you suspected, activeTypeDefs ends up being null --since the helper function looks up in the //local// repository, and in the coco environment the server cocoMDS4 is a member of the cohort, and is where the asset search UI is targetted at, but it does not have a repository of it's own. This is a perfectly valid configuration. We have many types, and a number of real assets -- and indeed even with this error, the UI is able to show search results (search for 'csv' for example)

As such the PR helps to mask the error, so may be worth merging, but there's a more fundamental change here needed so that the supported types can be queried properly.

This is one of the benefits of having the coco environment - consistency, and a typical 'enterprise' environment with multiple cohort numbers having differing roles.

But equally, running in other diverse configurations is important too so we can catch these kind of issues!

[danielaotelea]

Asset Catalog OMAS, should return the known type defs from that service, instead of the active type defs. I need to replace the getActiveTypeDefs with knownTypdeDefs because the server may runs with OMASes configuration, but without a local repository. I open a new PR for that.

[planetf1]

Thanks @danielaotelea will test that PR. From inspection looks good. It will be very useful to have this in 1.4 especially for the notebooks since we document asset catalog omas as new in 1.4

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 20 days if no further activity occurs. Thank you for your contributions.

[planetf1]

This is fixed now and can be closed