odpi / egeria-connector-hivemetastore

Egeria repository connector for Hive metastore. Initially targeted at spark metadata
Apache License 2.0
10 stars 5 forks source link

Catching class loading connection (Hadoop Configuration) #113

Closed planetf1 closed 1 year ago

planetf1 commented 1 year ago

When testing the HMS connector I an issue which did not result in any Audit log entries:

Class loading error

Due to a build error, I had a problem with the connector hitting a class loader exception when issuing

//populatetheHiveconfigurationfortheHMSclient.
Configurationconf=newConfiguration();

The error was in trying to load from javax.xml

Whilst the build error (during refactoring) is fixed, this exception results in an 'Internal server error' and no output on the audit log.

It's possible other similar circumstances could cause an issue

planetf1 commented 1 year ago

Also noticed this when accidentally running the standard connector against IBM Data Engine.

Again, only an internal server error with no other information logged.

Enabling debug showed the issue was a missing thrift library, but crucially this error never surfaced to the audit log or user

2023-02-27 08:49:10.029 DEBUG 54946 --- [nio-9443-exec-6] o.a.c.loader.WebappClassLoaderBase       :   --> Returning 'jar:file:/Users/jonesn/ibm-HMS/testplatform/lib/egeria-connector-hivemetastore-1.0-SNAPSHOT-jar-with-dependencies.jar!/core-default.xml'
2023-02-27 08:49:10.057 DEBUG 54946 --- [nio-9443-exec-6] o.a.c.loader.WebappClassLoaderBase       : getResource(core-site.xml)
2023-02-27 08:49:10.057 DEBUG 54946 --- [nio-9443-exec-6] o.a.c.loader.WebappClassLoaderBase       :   Delegating to parent classloader org.springframework.boot.loader.LaunchedURLClassLoader@7d9d1a19
2023-02-27 08:49:10.059 DEBUG 54946 --- [nio-9443-exec-6] o.a.c.loader.WebappClassLoaderBase       :   --> Resource not found, returning null
2023-02-27 08:49:10.065 DEBUG 54946 --- [nio-9443-exec-6] o.s.web.servlet.DispatcherServlet        : Failed to complete request: org.springframework.web.util.NestedServletException: Handler dispatch failed; nested exception is java.lang.NoClassDefFoundError: org/odpi/openmetadata/shaded/apache/thrift/transport/TFramedTransport
2023-02-27 08:49:10.070 ERROR 54946 --- [nio-9443-exec-6] o.a.c.c.C.[.[.[/].[dispatcherServlet]    : Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Handler dispatch failed; nested exception is java.lang.NoClassDefFoundError: org/odpi/openmetadata/shaded/apache/thrift/transport/TFramedTransport] with root cause

java.lang.ClassNotFoundException: org.odpi.openmetadata.shaded.apache.thrift.transport.TFramedTransport
    at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:445) ~[na:na]
    at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:587) ~[na:na]
    at org.springframework.boot.loader.LaunchedURLClassLoader.loadClass(LaunchedURLClassLoader.java:151) ~[server-chassis-spring-3.15.jar:na]
    at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:520) ~[na:na]
    at org.odpi.egeria.connectors.hms.eventmapper.HMSOMRSEventProducer.connectTo3rdParty(HMSOMRSEventProducer.java:142) ~[egeria-connector-hivemetastore-1.0-SNAPSHOT-jar-with-dependencies.jar:na]
    at org.odpi.egeria.connectors.hms.eventmapper.OMRSDatabasePollingRepositoryEventMapper.start(OMRSDatabasePollingRepositoryEventMapper.java:126) ~[egeria-connector-hivemetastore-1.0-SNAPSHOT-jar-with-dependencies.jar:na]
github-actions[bot] commented 1 year ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 20 days if no further activity occurs. Thank you for your contributions.