exonianp
commented
3 years ago Just to add that I tried to replicate the issue with all three available OpenJDKs (8 and 15) and the same issue exists in all.
Closed exonianp closed 3 years ago
exonianp
commented
3 years ago Just to add that I tried to replicate the issue with all three available OpenJDKs (8 and 15) and the same issue exists in all.
planetf1
commented
3 years ago I suspect this is an error with opening the truststore and probably not platform specific
See https://github.com/odpi/egeria/issues/3705 - we did close that after updating the docs at https://egeria-project.org/open-metadata-implementation/admin-services/docs/user/omag-server-platform-transport-level-security.html
The trustore is found if you run from our assemblies (open-metadata-distribution/open-metadata-assemblies/target etc..) and launch with 'java -jar server/chassis jar as we then find truststore.p12. This is also how we launch it within docker (and hence docker compose/k8s)
Similarly in the dev environment we find the truststore in the current directory
For any other layout it's too easy to fall into this situation
The docs and/or error handling are not good enough. We should
Can you retry, but specify the truststore specifically as per the TLS docs? You can use our shipped certs etc or generate your own. Further feedback on the docs would be really useful...
exonianp
commented
3 years ago 
Thank you the issue has been sorted now with running the following command (via ubuntu 20.04 shell on Windows Server): dimitrisv@METAPOLL:/mnt/c/Projects/egeria$ java -Dserver.ssl.key-store=open-metadata-implementation/server-chassis/server-chassis-spring/src/main/resources/keystore.p12 -Dserver.ssl.key-alias=tomcat -Dserver.ssl.key-store-password=egeria -Dserver.ssl.trust-store=open-metadata-implementation/server-chassis/server-chassis-spring/src/main/resources/truststore.p12 -Dserver.ssl.trust-store-password=egeria -Djavax.net.ssl.keyStore=open-metadata-implementation/server-chassis/server-chassis-spring/src/main/resources/keystore.p12 -Djavax.net.ssl.keyStorePassword=egergia -Djavax.net.ssl.trustStore=open-metadata-implementation/server-chassis/server-chassis-spring/src/main/resources/trusstore.p12 -Djavax.net.ssl.trustStorePassword=egeria -jar open-metadata-implementation/server-chassis/server-chassis-spring/target/server-chassis-spring-2.6-SNAPSHOT.jar
Just to mention that your sample bash script the trust store is defined as the keystore (btw both files are identical in size) and that reference in the path /build/libs/ does not exist anymore so I replaced it with target.
Now the server runs and can connect to it but gives an error so probably I will have to now check other configuration options.
Indeed it will be great if the installation docs are even more explicit. You have years of work in your project and I understand that the onboarding in a day can be challenging. However, one of the reasons why I opted to go for the local install (and not for the Docker) was simply because I wanted to see how the installation behaves in the same OS where I have installed my MS SQL and Neo4j servers that will actually provide the data I intent to share with egeria.
My scope of work in to use a platform like egeria for the storage of semantic triples (subject - predicate - object) that actually describe code (e.g. function X updates this field or function Y calls function X) that we currently visualize in graph databases (Neo4j and OrientDB). It is a concept that it is not unrelated to @mandy-chessell's past work re: the ethical framework. The point is that Egeria can be that great repository we are looking for in order to both act as data-dictionary as well as do our coarse "searches" and to then use our graphDB infrastructure to perform fine searches (as well as run algorithms in our graphs in order to get various metrics). As we are having millions of nodes and relationships to check, a graph db by itself is not optimal for this coarse searches.
To make a long story shorter, many of us interested in your work but, we may view it from the user or extender of your original scope perspectives, rather then becoming involved with the core of your effort. Plus, we may not be the java experts that you are. For example I used to code in java many years ago. For example, I am more involved with python (and various ML and Image Processing modules) and cypher; so I am not the pro Java developer I used to be. So if you ask me to build something and read it I can do it, but I am not following the latest developments. Therefore I think that it is imperative for you to invest a little bit more time in assisting the onboarding of users like me. I mean when I looked at the options, I said to myself that "since the local installation is just a build, a copy and a run, I rather go for it". But, even the issue with the certificates should be part of the installation process as you run your own instance of Tomcat in the project - thus you should look after the TLS issues. Similarly, I guess is the case with kafka. It is not evident if kafka is a pre-requisite in the system (thus we need more detailed info on how to connect with it), or if it is part of the bundle and fully configured.
Having said all this, I will invest two-three days to go after your onboarding tutorials and videos to get a deeper understanding - as well as learning the various modules that need to be configured and only then I will bother you again :). But, if you have any more direct advice on the configuration required please let me know ;).
A final suggestion is to integrate some search functionality in your site (and possibly in your project by bundling a tool like elasticsearch) and a more hierarchical menu structure especially in anything that deals with instructions/installation and configuration steps.
planetf1
commented
3 years ago Many thanks for the detailed feedback. Please keep it coming! I'm glad you were able to get up and running.
Will definitely take a little time to look at specific improvements we can make to help onboard new users. I totally agree with the point about making it easy for new people who just want to use/run egeria, rather than get involved in the actual development.
Currently I hope (!) our docker-compose and helm (k8s) charts are relatively simple to get going - as long as you have docker/k8s .. They do bundle together the configuration, kafka, etc to get you running quickly. They are though currently more targetted at the tutorial use case, and less so at getting a custom environment running. We're considering spending more time on this transition ie how to just run egeria without all the notebooks/coco pharma configuration, and also for k8s have work underway on an operator (though that in itself is a more specific k8s approach)
The local case is harder in terms of steps, but we need to do better in providing a simple list of steps - at least for the initial setup/run. There's been some talk around what could be scripted/packaged - though that tends to become platform specific and subject to the differences between users/machines
planetf1
commented
3 years ago On search, in part I presume you mean the documentation? Understand. We currently use github pages and I don't believe this offers search directly, though google can help as can the structure we chose to put in place - so should review that. Searching directly on the repo and then filtering by file type 'markdown' is something I find useful, but realise it's not so end user friendly. We also have work underway to add javadoc content and publish. We'll think more about this.....
exonianp
commented
3 years ago Thank you Nigel!
With the previous moaning and groaning :) I forgot to mention the obvious. That the platform has been installed successfully (in spite of the error in the landing page)
I would rather skip the docker. As said I wanted to test the whole platform coexisting with our other elements (neo4j, mssql) in the same kit. We may assist in the development here as well (with colleagues who are hands on Java experts) but for the time being we are just evaluating how we can combine your effort/infrastructure with our (meta) data (meta) analysis needs.
Your architecture is perfect. Our concern is if/how we can use it to combine data-dictionaries, with automatic code analysis (the triplets I mentioned before) and the visualisation of information. Also the part of extensibility/flexibility (e.g. what if we want to use our own Neo4j as our graphdb and not your suggested solution).
As for search I was referring to the site page (not github) and the platform itself. I have yet to reach the stage to get to know how you search within the platform. I am looking forward to installing the coco demo soon.
Keep up the good work!
planetf1
commented
3 years ago Just going to leave this open until I'm sure we've answered all the points made here
planetf1
commented
3 years ago Just to summarise :
What I think will be easiest on the other issues is to raise a specific issue on any problem you are facing - for example 'how do I get started writing a neo4j graph repo connector' if you can't find the docs easily ( https://egeria.odpi.org/open-metadata-implementation/adapters/open-connectors/repository-services-connectors/open-metadata-collection-store-connectors/docs/ ? ). There absolutely is no need to use the graph repo we provide .... the architecture is very pluggable. It makes it more actionable. For higher level discussion our slack channels I hope are useful.
I'll close the issue here for now but look forward to more questions/issues in the future :-)
Having successfully built the egeria on Windows
[INFO] Digital Architecture OMAS FVT ...................... SUCCESS [01:29 min] [INFO] Subject Area OMAS FVT .............................. SUCCESS [04:31 min] [INFO] Open Metadata Open Types Test Resources ............ SUCCESS [ 0.297 s] [INFO] Open Types Test generator .......................... SUCCESS [ 7.469 s] [INFO] Open Types Test execution .......................... SUCCESS [ 43.187 s] [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 52:47 min [INFO] Finished at: 2020-12-03T12:11:47+02:00 [INFO] ------------------------------------------------------------------------
When it came to running it the following error is observed: c:\Projects\egeria-install\egeria-omag-2.6-SNAPSHOT\server>java -Dserver.port=9443 -jar server-chassis-spring-2.6-SNAPSHOT.jar ODPi Egeria
/ \ / |/ // | / __/ / / _ _ _ / \ / / _ / / / / _ / / / // /|/ // /| | / / \ \ / _ \ / /| | / // _ \ / / / // // // | / \ / / / | / // || | / // // / / // |/ /_/ / _/ // // / | |/ // // / / // // / \ / / / // / // / / / / / __/// //// ||__/ /__/ _//_/ |/ \/// // // _/\///// \/// // /_/
:: Powered by Spring Boot (v2.4.0) ::
12:37:20.728 [main] INFO o.s.b.w.e.tomcat.TomcatWebServer - Tomcat initialized with port(s): 9443 (https) 12:37:29.028 [main] ERROR o.s.boot.SpringApplication - Application run failed org.springframework.context.ApplicationContextException: Failed to start bean 'webServerStartStop'; nested exception is org.springframework.boot.web.server.WebServerException: Unable to start embedded Tomcat server at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:181) at org.springframework.context.support.DefaultLifecycleProcessor.access$200(DefaultLifecycleProcessor.java:54) at org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:356) at java.base/java.lang.Iterable.forEach(Iterable.java:75) at org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:155) at org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(DefaultLifecycleProcessor.java:123) at org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:942) at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:591) at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:144) at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:767) at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:759) at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:426) at org.springframework.boot.SpringApplication.run(SpringApplication.java:326) at org.springframework.boot.SpringApplication.run(SpringApplication.java:1309) at org.springframework.boot.SpringApplication.run(SpringApplication.java:1298) at org.odpi.openmetadata.serverchassis.springboot.OMAGServerPlatform.main(OMAGServerPlatform.java:93) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49) at org.springframework.boot.loader.Launcher.launch(Launcher.java:107) at org.springframework.boot.loader.Launcher.launch(Launcher.java:58) at org.springframework.boot.loader.PropertiesLauncher.main(PropertiesLauncher.java:467) Caused by: org.springframework.boot.web.server.WebServerException: Unable to start embedded Tomcat server at org.springframework.boot.web.embedded.tomcat.TomcatWebServer.start(TomcatWebServer.java:229) at org.springframework.boot.web.servlet.context.WebServerStartStopLifecycle.start(WebServerStartStopLifecycle.java:43) at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:178) ... 23 common frames omitted Caused by: java.lang.IllegalArgumentException: standardService.connector.startFailed at org.apache.catalina.core.StandardService.addConnector(StandardService.java:231) at org.springframework.boot.web.embedded.tomcat.TomcatWebServer.addPreviouslyRemovedConnectors(TomcatWebServer.java:282) at org.springframework.boot.web.embedded.tomcat.TomcatWebServer.start(TomcatWebServer.java:213) ... 25 common frames omitted Caused by: org.apache.catalina.LifecycleException: Protocol handler start failed at org.apache.catalina.connector.Connector.startInternal(Connector.java:1067) at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183) at org.apache.catalina.core.StandardService.addConnector(StandardService.java:227) ... 27 common frames omitted Caused by: java.lang.IllegalArgumentException: c:\Projects\egeria-install\egeria-omag-2.6-SNAPSHOT\server\truststore.p12 (The system cannot find the file specified) at org.apache.tomcat.util.net.AbstractJsseEndpoint.createSSLContext(AbstractJsseEndpoint.java:99) at org.apache.tomcat.util.net.AbstractJsseEndpoint.initialiseSsl(AbstractJsseEndpoint.java:71) at org.apache.tomcat.util.net.NioEndpoint.bind(NioEndpoint.java:216) at org.apache.tomcat.util.net.AbstractEndpoint.bindWithCleanup(AbstractEndpoint.java:1141) at org.apache.tomcat.util.net.AbstractEndpoint.start(AbstractEndpoint.java:1227) at org.apache.coyote.AbstractProtocol.start(AbstractProtocol.java:603) at org.apache.catalina.connector.Connector.startInternal(Connector.java:1064) ... 29 common frames omitted Caused by: java.io.FileNotFoundException: c:\Projects\egeria-install\egeria-omag-2.6-SNAPSHOT\server\truststore.p12 (The system cannot find the file specified) at java.base/java.io.FileInputStream.open0(Native Method) at java.base/java.io.FileInputStream.open(FileInputStream.java:219) at java.base/java.io.FileInputStream.(FileInputStream.java:157)
at java.base/java.io.FileInputStream.(FileInputStream.java:112)
at java.base/sun.net.www.protocol.file.FileURLConnection.connect(FileURLConnection.java:86)
at java.base/sun.net.www.protocol.file.FileURLConnection.getInputStream(FileURLConnection.java:184)
at org.apache.catalina.startup.CatalinaBaseConfigurationSource.getResource(CatalinaBaseConfigurationSource.java:121)
at org.apache.tomcat.util.net.SSLUtilBase.getStore(SSLUtilBase.java:197)
at org.apache.tomcat.util.net.SSLHostConfig.getTruststore(SSLHostConfig.java:722)
at org.apache.tomcat.util.net.SSLUtilBase.getTrustManagers(SSLUtilBase.java:423)
at org.apache.tomcat.util.net.SSLUtilBase.createSSLContext(SSLUtilBase.java:246)
at org.apache.tomcat.util.net.AbstractJsseEndpoint.createSSLContext(AbstractJsseEndpoint.java:97)
... 35 common frames omitted
Just for your information to mention that Tomcat 9.0.40 is already installed on my kit for another application.
Java version is as follows:
java --version openjdk 11.0.9.1 2020-11-04 OpenJDK Runtime Environment AdoptOpenJDK (build 11.0.9.1+1) OpenJDK 64-Bit Server VM AdoptOpenJDK (build 11.0.9.1+1, mixed mode)As it appears that I may be the only tester on Windows, if it can be of any help to the team I am happy to provide you with access in my aws test server, for further investigation and fix.