neo4j / neo4j-browser

Neo4j Browser is the general purpose user interface for working with Neo4j. Query, visualize, administrate and monitor the database.
https://neo4j.com
GNU General Public License v3.0
690 stars 347 forks source link

Browser 7474 not available in Neo4j 3.1.0 embedded #342

Closed obrienlabs closed 6 years ago

obrienlabs commented 7 years ago

Having issues bridging bolt and http to enable the browser to come up under 7474 like it did for 2.3

See other tracking issue

https://github.com/neo4j/neo4j/issues/7608 2.14 of http://neo4j.com/docs/java-reference/current/#tutorials-java-embedded https://neo4j.com/developer/kb/how-do-i-enable-remote-https-access-with-neo4j-30x/

We are running Neo4j 3.1.0 as an embedded graph.db and also would like to enable the 7474 neo4j HTTP browser as we did in 2.3.7. Conf parameters are setting 7474 to HTTP from BOLT in 3.1.0, in 3.0.7 the port behaves like a bolt websocket port

Bottom line is:

3.0.7 brings up 7474 as a bolt port 3.1.0 does not bring up 7474 at all

In 3.1 the gui is different for the 3.0 procedure where a standalone browser is used to adjust the embedded graph.db - the bolt-http connector is still not up though.

screenshot 2016-12-17 23 43 56

Can you advise if the bolt+routing protocol workaround for 3.1 is valid - when it was put in to 3.1 the issue states "when in a causal cluster"

cl: Add support for the bolt+routing protocol when in a Causal Cluster https://github.com/neo4j/neo4j-browser/pull/322

To check - I switched back to 3.0.7 (deleted graph.db) - get different behaviour I get the same malformed websocket message I see in 7474 that I get if I access bolt 7687 directly in 3.1.0

likely because 7474 ends up configured with BOLT instead of HTTP

2016-12-18 15:03:26.189+0000 INFO  [o.n.b.v.r.Sessions] Bolt Server extension loaded.
2016-12-18 15:03:26.190+0000 INFO  [o.n.b.v.r.Sessions] Bolt enabled on 0.0.0.0:7687.
2016-12-18 15:03:26.190+0000 INFO  [o.n.b.v.r.Sessions] Bolt enabled on 0.0.0.0:7474.

This is with or without the bolt settings applied on the factory and running only with

        <entry key="dbms.connector.http.address" value="0.0.0.0:7474" />
        <entry key="dbms.connector.http.enabled" value="true" />
        <entry key="dbms.connector.http.type" value="HTTP" />

and with/without bolt settings applied

public class ExtendedHighlyAvailableGraphDatabaseFactory extends HighlyAvailableGraphDatabaseFactory {
    private HaMonitor haMonitor;
    private GraphDatabaseSettings.BoltConnector bolt = GraphDatabaseSettings.boltConnector( "0" );
    @Override
     protected GraphDatabaseBuilder.DatabaseCreator createDatabaseCreator(
                final File storeDir, final GraphDatabaseFactoryState state)  {
            return new GraphDatabaseBuilder.DatabaseCreator() {
                @Override
                public GraphDatabaseService newDatabase( final Map<String, String> config ) {
                    // section 2.14 of http://neo4j.com/docs/java-reference/current/#tutorials-java-embedded
//                    config.put( bolt.type.name(), "BOLT" );
//                    config.put( bolt.enabled.name(), "true" );
//                    config.put( bolt.address.name(), "0.0.0.0:7687" );

running

           <neo4j.version>3.0.7</neo4j.version>
            <!--neo4j.version>3.1.0</neo4j.version-->
   </properties>
  <dependencies>
      <dependency>
        <groupId>org.neo4j</groupId>
        <artifactId>neo4j-bolt</artifactId>
        <version>${neo4j.version}</version>
    </dependency> 

"not a WebSocket handshake request: missing upgrade"

A 400 this time Request URL:http://localhost:7474/ Request Method:GET Status Code:400 Bad Request Remote Address:127.0.0.1:7474

3.0.7
2016-12-18 15:03:26.189+0000 INFO  [o.n.b.v.r.Sessions] Bolt Server extension loaded.
2016-12-18 15:03:26.190+0000 INFO  [o.n.b.v.r.Sessions] Bolt enabled on 0.0.0.0:7687.
2016-12-18 15:03:26.190+0000 INFO  [o.n.b.v.r.Sessions] Bolt enabled on 0.0.0.0:7474.
2016-12-18 15:24:46.851+0000 INFO  [o.n.k.i.DiagnosticsManager] Kernel version: 3.0.7,844933cd439bde936b04e66f1c0ca8d617794a0a

obrienlabs-mbp15:_deployment michaelobrien$ netstat -vatn | grep 7474
tcp46      0      0  *.7474                 *.*                    LISTEN      131072 131072  35888      0
tcp6       0      0  ::1.7474               ::1.49967              TIME_WAIT   407223 146808  35888      0
tcp6       0      0  ::1.7474               ::1.49976              TIME_WAIT   407444 146808  35888      0
tcp6       0      0  ::1.7474               ::1.49978              TIME_WAIT   407223 146808  35888      0
tcp6       0      0  ::1.7474               ::1.49979              TIME_WAIT   407223 146808  35888      0
tcp6       0      0  ::1.7474               ::1.49980              TIME_WAIT   407223 146808  35888      0
tcp6       0      0  ::1.7474               ::1.49982              TIME_WAIT   407223 146808  35888      0
obrienlabs-mbp15:_deployment michaelobrien$ netstat -vatn | grep 8080
tcp4       0      0  127.0.0.1.8080         127.0.0.1.49948        FIN_WAIT_2  408104 146988  35888      0
tcp46      0      0  *.8080                 *.*                    LISTEN      131072 131072  35888      0
tcp6       0      0  ::1.8080               ::1.49961              TIME_WAIT   407402 146808  35888      0
obrienlabs-mbp15:_deployment michaelobrien$ netstat -vatn | grep 7687
tcp46      0      0  *.7687                 *.*                    LISTEN      131072 131072  35888      0
3.1.0
2016-12-18 15:27:02.128+0000 INFO  [o.n.b.v.r.WorkerFactory] Bolt Server extension loaded.
2016-12-18 15:27:02.128+0000 INFO  [o.n.b.v.r.WorkerFactory] Bolt enabled on 0.0.0.0:7687.
2016-12-18 15:27:05.457+0000 INFO  [o.n.k.i.DiagnosticsManager] Kernel version: 3.1.0,16a782b42d76ca37db72958eb2565cf6aa671a29

obrienlabs-mbp15:_deployment michaelobrien$ netstat -vatn | grep 7474
obrienlabs-mbp15:_deployment michaelobrien$ netstat -vatn | grep 8080
tcp6       0      0  ::1.8080               ::1.51243              ESTABLISHED 407402 146808  36204      0
tcp6       0      0  ::1.51243              ::1.8080               ESTABLISHED 407624 146808  24134      0
tcp4       0      0  127.0.0.1.8080         127.0.0.1.51236        ESTABLISHED 408104 146988  36204      0
tcp4     994      0  127.0.0.1.51236        127.0.0.1.8080         ESTABLISHED 408128 146988  13649      0
tcp46      0      0  *.8080                 *.*                    LISTEN      131072 131072  36204      0
tcp4       0      0  127.0.0.1.8080         127.0.0.1.51231        TIME_WAIT   408300 146988  36188      0
obrienlabs-mbp15:_deployment michaelobrien$ netstat -vatn | grep 7687
tcp46      0      0  *.7687                 *.*                    LISTEN      131072 131072  36204      0

//I'll try 3.2.0-alpha02 when it comes up on maven central as well.

The KernelExtensionFactory impls contain only one Bolt subclass that could setup the 7687 and 7474 connectors. If I force the http connector as type BOLT - I register both of them (7474 is then a websocket port) - but this is not a solution.


Debugging the GraphDatabase Initialization I see () implementations of KernelExtensionFactory
    @Override
    public Lifecycle newInstance( KernelContext context, Dependencies dependencies ) throws Throwable

each KernelExtensionFactory implements newInstance() 
BoltKernelExtension
Lucene* (3 for indexing)
Metrics*
OnlineBackup*
QueryLogger*
ShellServer*
Udc*

connectors  ArrayList<E>  (id=197)  
    elementData Object[10]  (id=208)    
        [0] SocketTransport  (id=211)   
            address ListenSocketAddress  (id=218)   
                hostname    "localhost" (id=278)    
                port    7687    
        [1] SocketTransport  (id=212)   
            address ListenSocketAddress  (id=235)   
                hostname    "localhost" (id=275)    
                port    7474    
public class BoltKernelExtension extends KernelExtensionFactory<BoltKernelExtension.Dependencies>

        if ( connectors.size() > 0 && !config.get( GraphDatabaseSettings.disconnected ) )
        {
            life.add( new NettyServer( scheduler.threadFactory( boltNetworkIO ), connectors ) );
            log.info( "Bolt Server extension loaded." );
            for ( ProtocolInitializer connector : connectors )
            {
                logService.getUserLog( WorkerFactory.class ).info( "Bolt enabled on %s.", connector.address() );

It looks like we need to be running the connector setup that occurs in NeoServer and its subclasses down to EnterpriseNeoServer where we setup the following - however this code only runs for non-embedded servers

              new Neo4jBrowserModule( webServer ),
        HttpConnector httpConnector = ClientConnectorSettings.httpConnector( config, ClientConnectorSettings.HttpConnector.Encryption.NONE )
                .orElseThrow( () ->
                        new IllegalArgumentException( "An HTTP connector must be configured to run the server" ) );
        httpListenAddress = config.get( httpConnector.listen_address );
        httpAdvertisedAddress = config.get( httpConnector.advertised_address );

I did run into an issue with SSL creation during OPTIONAL (the default) only when a certain logging parameter was set to DEBUG - I'll raise a separate issue on this. The fix was to set tls_level on the indexed 0 bolt property not directly on bolt.tls_level

      <entry key="dbms.connector.bolt.tls_level" value="DISABLED" /> <!--  no effect - missing index 0 -->
        <entry key="dbms.logs.debug.level" value="DEBUG"/
causes

java.lang.IllegalArgumentException: Failed to load any of the given libraries: [netty-tcnative-osx-x86_64, netty-tcnative]
    at io.netty.util.internal.NativeLibraryLoader.loadFirstAvailable(NativeLibraryLoader.java:173)
    at io.netty.handler.ssl.OpenSsl.loadTcNative(OpenSsl.java:380)
    at io.netty.handler.ssl.OpenSsl.<clinit>(OpenSsl.java:99)
    at io.netty.handler.ssl.SslContext.defaultProvider(SslContext.java:117)
    at io.netty.handler.ssl.SslContext.defaultServerProvider(SslContext.java:104)
    at io.netty.handler.ssl.SslContext.newServerContextInternal(SslContext.java:404)
    at io.netty.handler.ssl.SslContextBuilder.build(SslContextBuilder.java:402)
    at org.neo4j.bolt.BoltKernelExtension.createSslContext(BoltKernelExtension.java:224)
    at org.neo4j.bolt.BoltKernelExtension.lambda$newInstance$0(BoltKernelExtension.java:182)

fixed with
                    config.put( bolt.encryption_level.name(), "DISABLED" );
which renders in jconsole as 
dbms.connector.0.bolt.tls_level

Just to verify our custom 4 level constructor override of GraphDatabaseService to load our custom HA listener is causing us connector deviation I ran a very basic embedded setup - same issue

        GraphDatabaseSettings.BoltConnector bolt = GraphDatabaseSettings.boltConnector( "0" );
        GraphDatabaseService graphDb = new GraphDatabaseFactory()
        .newEmbeddedDatabaseBuilder( new File("/ec2-user/graph2.db" ))
        .setConfig(bolt.type, "BOLT" )
        .setConfig( bolt.enabled, "true" )
        .setConfig( bolt.address, "0.0.0.0:7687" )
        .setConfig( bolt.encryption_level, "DISABLED" ) //dbms.connector.0.tls_level
        .setConfig("dbms.connector.http.address","0.0.0.0:7474")
        .setConfig("dbms.connector.http.enabled","true" )
        .setConfig("dbms.connector.http.type", "HTTP" )
        .setConfig("dbms.connector.http.tls_level", "DISABLED")
        .newGraphDatabase();

graphDb GraphDatabaseFacade (id=198)
community single [/ec2-user/graph2.db]

TODO: I still have a couple experiments and debugging I can do to try to force 7474 port creation on the embedded instance. Sad because 7474 works so well on a standalone server via the sh command

[root@obrien bin]# ./neo4j console
Starting Neo4j.
2016-12-18 23:25:26.073+0000 INFO  No SSL certificate found, generating a self-signed certificate..
2016-12-18 23:25:26.499+0000 INFO  Starting...
2016-12-18 23:25:27.227+0000 INFO  Bolt enabled on localhost:7687.
2016-12-18 23:25:27.241+0000 INFO  Initiating metrics...
2016-12-18 23:25:30.054+0000 INFO  Started.
2016-12-18 23:25:30.239+0000 INFO  Mounted REST API at: /db/manage
2016-12-18 23:25:31.331+0000 INFO  Remote interface available at http://localhost:7474/

Attaching small maven war eclipse project used to test biometric.nbi.web-0.0.11-SNAPSHOT_eclipse_maven_src.zip

obrienlabs commented 7 years ago

Anne's/J.F.'s workaround using 2 neo4j servers is good - I verified it on my setup - I get the same NPE as well.

Use the browser from a standalone to run queries against the embedded servers's bolt 7688 address without having to shutdown either graph.db

1) graph1.db managed by embedded neo4j (nsp live db) 2) graph2.db managed by standalone neo4j (app db - stub in the context of nsp) 3) launch embedded neo4j on bolt 7688 dbms.connector.bolt.address=127.0.0.1:7688 4) launch standalone neo4j on bolt 7687 (default) 5) in http://localhost:7474 point the bolt address to 7688 (didn't realize we could edit this greyed box) 6) now queries from 7474 will run against the embedded graph1.db instead of the standalone graph2.db Going forward: Currently a pure embedded server cannot instantiate an http port (the code in in NeoServer) - we would need to revert back to the Wrapping class removed in 3.x No effect

{noformat}

    <entry key="dbms.connector.http.enabled" value="true" />
    <entry key="dbms.connector.http.type" value="HTTP" />

{noformat}

Ideally a procedure to wrap the embedded server' graphDatabaseFacade so that 7474 can be exposed via NeoServer code would be good in the future

The jmx exception does not block querying the db but should be looked at

{noformat}
match(n) return(n);
{noformat}

{noformat}
2016-12-19 16:46:27.595+0000 ERROR [o.n.b.v.r.ErrorReporter] Client triggered an unexpected error [UnknownError]: Exception invoking method writeBufSize. See debug.log for more details, reference dcc80076-665c-4cde-be3b-041c9253647c.
2016-12-19 16:46:27.596+0000 ERROR [o.n.b.v.r.ErrorReporter] Client triggered an unexpected error [UnknownError]: Exception invoking method writeBufSize, reference dcc80076-665c-4cde-be3b-041c9253647c. Exception invoking method writeBufSize
javax.management.RuntimeOperationsException: Exception invoking method writeBufSize
    at org.apache.tomcat.util.modeler.BaseModelMBean.getAttribute(BaseModelMBean.java:196)
    at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:647)
    at com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(JmxMBeanServer.java:678)
    at org.neo4j.kernel.builtinprocs.JmxQueryProcedure.toNeo4jValue(JmxQueryProcedure.java:131)
    at org.neo4j.kernel.builtinprocs.JmxQueryProcedure.toNeo4jValue(JmxQueryProcedure.java:119)

{noformat}
override_bolt_7687_to7688_run_browser_across_2_neo4j_servers_screenshot 2016-12-19 11 47 28
obrienlabs commented 7 years ago

It turns out with a bit of help from the following post we can still bring up a wrapping NeoServer around our embedded mode Neo4j 3.1.0 server. This code however is not integrated yet with our HA listener overrides on the constructor tree we bootstrap in Spring - so is a POC.

With this NeoServer wrapper code - we don’t need the secondary workaround where we bring up a 2nd Neo4j server mode instance, bring up its’ 7474 browser and modify the bolt address (default 7687) in the config to point to the exposed (7688) embedded mode graph.db - without bringing either Neo4j server down as long as they point to different graph.db instances - in effect sharing the 7474 browser as a bolt client.

Disclaimer: I believe a proper architecture involves a clustered set of server mode bolt enabled Neo4j servers. This undocumented/internal API is used as a temporary workaround to get the neo4j browser up on an embedded Neo4j server in 3.1.0 as it used to work under the WrappingNeoServerBootstrapper in 2.3 so we can continue to use in-jvm Dijkstra algos until we are able to rework/benchmark the same or greater traversal speed via a bolt enabled cluster.

FrontController/spring service bean starts graph

2016-12-20 01:28:04.186+0000 INFO  Starting...
2016-12-20 01:28:04.533+0000 INFO  Bolt enabled on localhost:7687.
2016-12-20 01:28:04.534+0000 INFO  Initiating metrics...
2016-12-20 01:28:06.091+0000 INFO  Started.
2016-12-20 01:28:07.009+0000 INFO  Remote interface available at http://localhost:7575/
<state>community single [/ec2-user/graph2.db] : community single [/ec2-user/graph2.db/data/databases/graph.db]</state>

after opening browser

        // http://stackoverflow.com/questions/30074232/replacement-for-deprecated-wrappingneoserverbootstrapper
        // Warning: internal/unsupported API - NeoServer wrappers are usually only run in  server mode - not embedded
        ServerBootstrapper serverBootstrapper = new EnterpriseBootstrapper();//CommunityBootstrapper();
        NeoServer directNeoServer = serverBootstrapper.getServer();
        int i = serverBootstrapper.start(new File("/ec2-user"), // will resolve to /ec2-user/data/databases/graph.db 
                Optional.of(new File("/Users/michaelobrien/Documents/Neo4j/.neo4j.conf")), //  stub only
                                Pair.of("dbms.connector.http.address","0.0.0.0:7575"),
                                Pair.of("dbms.connector.http.enabled","true" ),
                                Pair.of("dbms.connector.http.type", "HTTP" ),
                                Pair.of("dbms.connector.http.tls_level", "DISABLED")
                                // BOLT not advised if we run an embedded Neo4j DB inside a NeoServer wrapper
                                // BOLT should be enabled if you run a standalone server's browser against this embedded bolt port
                                // WebSocket connection failure. Due to security constraints in your web browser, the reason for the failure is not available to this Neo4j Driver. Please use your browsers development console to determine the root cause of the failure. Common reasons include the database being unavailable, using the wrong connection URL or temporary network problems. If you have enabled encryption, ensure your browser is configured to trust the certificate Neo4j is configured to use. WebSocket `readyState` is: 3
                                ,Pair.of("dbms.connector.bolt.address","0.0.0.0:7688"),
                                Pair.of("dbms.connector.bolt.enabled","true" ),
                                Pair.of("dbms.connector.bolt.type", "HTTP" ),
                                Pair.of("dbms.connector.bolt.tls_level", "DISABLED")                            
                        );
        NeoServer neoServer = serverBootstrapper.getServer();
        GraphDatabaseService graph = neoServer.getDatabase().getGraph();  

Debugging we see that we get the proper Jetty server available usually only to server mode Neo4j not embedded mode.

AbstractNeoServer.<clinit>() line: 114  
EnterpriseBootstrapper.createNeoServer(Config, GraphDatabaseDependencies, LogProvider) line: 45 
EnterpriseBootstrapper(ServerBootstrapper).start(File, Optional<File>, Pair<String,String>...) line: 90 
FrontController.processGraph3(HttpServletRequest, HttpServletResponse, PrintWriter) line: 161   

CommunityNeoServer.<clinit>() line: 54  
EnterpriseBootstrapper.createNeoServer(Config, GraphDatabaseDependencies, LogProvider) line: 45 
EnterpriseBootstrapper(ServerBootstrapper).start(File, Optional<File>, Pair<String,String>...) line: 90 
FrontController.processGraph3(HttpServletRequest, HttpServletResponse, PrintWriter) line: 161   

Config.get(Setting<T>) line: 137    
EnterpriseNeoServer(AbstractNeoServer).<init>(Config, Database$Factory, GraphDatabaseFacadeFactory$Dependencies, LogProvider) line: 159 
EnterpriseNeoServer(CommunityNeoServer).<init>(Config, Database$Factory, GraphDatabaseFacadeFactory$Dependencies, LogProvider) line: 69 
EnterpriseNeoServer.<init>(Config, GraphDatabaseFacadeFactory$Dependencies, LogProvider) line: 105  
EnterpriseBootstrapper.createNeoServer(Config, GraphDatabaseDependencies, LogProvider) line: 45 
EnterpriseBootstrapper(ServerBootstrapper).start(File, Optional<File>, Pair<String,String>...) line: 90 
FrontController.processGraph3(HttpServletRequest, HttpServletResponse, PrintWriter) line: 161   

Jetty9WebServer.<init>(LogProvider, Config) line: 135   
EnterpriseNeoServer(CommunityNeoServer).createWebServer() line: 89  
EnterpriseNeoServer.createWebServer() line: 131 
EnterpriseNeoServer(AbstractNeoServer).init() line: 181 
EnterpriseNeoServer(AbstractNeoServer).start() line: 196    
EnterpriseBootstrapper(ServerBootstrapper).start(File, Optional<File>, Pair<String,String>...) line: 91 
FrontController.processGraph3(HttpServletRequest, HttpServletResponse, PrintWriter) line: 161   

GraphDatabaseFacadeFactory.initFacade(File, Map<String,String>, Dependencies, GraphDatabaseFacade) line: 141    
EnterpriseGraphDatabase.<init>(File, Map<String,String>, Dependencies) line: 36 
EnterpriseNeoServer.lambda$static$1(Config, GraphDatabaseFacadeFactory$Dependencies) line: 90   
1274309106.newGraphDatabase(Config, GraphDatabaseFacadeFactory$Dependencies) line: not available    
LifecycleManagingDatabase.start() line: 89  
LifeSupport$LifecycleInstance.start() line: 433 
LifeSupport.start() line: 107   
EnterpriseNeoServer(AbstractNeoServer).start() line: 199    
EnterpriseBootstrapper(ServerBootstrapper).start(File, Optional<File>, Pair<String,String>...) line: 91 
FrontController.processGraph3(HttpServletRequest, HttpServletResponse, PrintWriter) line: 161   

Jetty9WebServer.loadJAXRSResource(SessionManager, String, JaxRsServletHolderFactory) line: 536  
Jetty9WebServer.loadJAXRSClasses(SessionManager, String) line: 522  
Jetty9WebServer.loadAllMounts() line: 409   
Jetty9WebServer.start() line: 171   
EnterpriseNeoServer(AbstractNeoServer).startWebServer() line: 326   
EnterpriseNeoServer(AbstractNeoServer).start() line: 218    
EnterpriseBootstrapper(ServerBootstrapper).start(File, Optional<File>, Pair<String,String>...) line: 91 
FrontController.processGraph3(HttpServletRequest, HttpServletResponse, PrintWriter) line: 161   

serverBootstrapper  EnterpriseBootstrapper  (id=299)    
    dependencies    GraphDatabaseDependencies  (id=347) 
    log FormattedLog  (id=573)  
    server  EnterpriseNeoServer  (id=357)   
    serverAddress   "0.0.0.0:7575" (id=575) 
    shutdownHook    ServerBootstrapper$1  (id=577)  

neoServer   EnterpriseNeoServer  (id=357)   
….  
    webServer   Jetty9WebServer  (id=406)   

from NeoServer
graph   EnterpriseGraphDatabase  (id=417)   
    contextFactory  Neo4jTransactionalContextFactory  (id=1119) 
    defaultTransactionTimeout   0   
    indexManager    Suppliers$1  (id=1121)  
    nodeActions StandardNodeActions  (id=1123)  
    relActions  StandardRelationshipActions  (id=1128)  
    schema  SchemaImpl  (id=1132)   
    spi ClassicCoreSPI  (id=425)    

Without NeoServer
graphDb GraphDatabaseFacade  (id=150)   
    contextFactory  Neo4jTransactionalContextFactory  (id=1182) 
    defaultTransactionTimeout   0   
    indexManager    Suppliers$1  (id=1183)  
    nodeActions StandardNodeActions  (id=1184)  
    relActions  StandardRelationshipActions  (id=1208)  
    schema  SchemaImpl  (id=1219)   
    spi ClassicCoreSPI  (id=160)    
2016-12-20 02:15:02.825+0000 INFO  Starting...
2016-12-20 02:15:03.304+0000 INFO  Bolt enabled on 0.0.0.0:7688.
2016-12-20 02:15:03.316+0000 INFO  Initiating metrics...
2016-12-20 02:15:06.232+0000 INFO  Started.
2016-12-20 02:15:06.412+0000 INFO  Mounted REST API at: /db/manage
2016-12-20 02:15:07.103+0000 INFO  Remote interface available at http://localhost:7575/
<state> : enterprise single [/ec2-user/data/databases/graph.db]</state>

settings and logs

cat /ec2-user/logs/debug.log
2016-12-20 02:15:03.225+0000 INFO  [o.n.k.i.DiagnosticsManager] dbms.connector.http.enabled=true
2016-12-20 02:15:03.225+0000 INFO  [o.n.k.i.DiagnosticsManager] dbms.security.auth_enabled=true
2016-12-20 02:15:03.226+0000 INFO  [o.n.k.i.DiagnosticsManager] dbms.connector.bolt.type=HTTP
2016-12-20 02:15:03.226+0000 INFO  [o.n.k.i.DiagnosticsManager] unsupported.dbms.directories.neo4j_home=/ec2-user
2016-12-20 02:15:03.226+0000 INFO  [o.n.k.i.DiagnosticsManager] dbms.directories.import=import
2016-12-20 02:15:03.226+0000 INFO  [o.n.k.i.DiagnosticsManager] dbms.connector.bolt.tls_level=DISABLED
2016-12-20 02:15:03.226+0000 INFO  [o.n.k.i.DiagnosticsManager] dbms.connector.http.type=HTTP
2016-12-20 02:15:03.226+0000 INFO  [o.n.k.i.DiagnosticsManager] dbms.logs.http.enabled=false
2016-12-20 02:15:03.226+0000 INFO  [o.n.k.i.DiagnosticsManager] unsupported.dbms.edition=enterprise
2016-12-20 02:15:03.226+0000 INFO  [o.n.k.i.DiagnosticsManager] dbms.connector.bolt.address=0.0.0.0:7688
2016-12-20 02:15:03.226+0000 INFO  [o.n.k.i.DiagnosticsManager] dbms.connector.http.address=0.0.0.0:7575
2016-12-20 02:15:03.226+0000 INFO  [o.n.k.i.DiagnosticsManager] dbms.connector.bolt.enabled=true
2016-12-20 02:15:03.226+0000 INFO  [o.n.k.i.DiagnosticsManager] dbms.connector.http.tls_level=DISABLED
2016-12-20 02:15:03.304+0000 INFO  [o.n.b.v.r.WorkerFactory] Bolt Server extension loaded.
2016-12-20 02:15:03.304+0000 INFO  [o.n.b.v.r.WorkerFactory] Bolt enabled on 0.0.0.0:7688.
2016-12-20 02:15:04.687+0000 INFO  [o.n.k.i.DiagnosticsManager] Kernel version: 3.1.0,16a782b42d76ca37db72958eb2565cf6aa671a29

These errors are non-fatal related to JMX and can be fixed

2016-12-20 02:17:30.759+0000 ERROR [o.n.b.v.r.ErrorReporter] Client triggered an unexpected error [UnknownError]: Exception invoking method writeBufSize, reference 59659a03-461e-4532-9b98-7afe3a2e5704. Exception invoking method writeBufSize
javax.management.RuntimeOperationsException: Exception invoking method writeBufSize
    at org.apache.tomcat.util.modeler.BaseModelMBean.getAttribute(BaseModelMBean.java:196)
    at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:647)
    at com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(JmxMBeanServer.java:678)
    at org.neo4j.kernel.builtinprocs.JmxQueryProcedure.toNeo4jValue(JmxQueryProcedure.java:131)
Caused by: java.lang.NullPointerException
    at org.apache.tomcat.util.net.SocketProperties.getTxBufSize(SocketProperties.java:300)

7575 http and 7688 bolt ports

obrienlabs-mbp15:_deployment michaelobrien$ netstat -vatn | grep 7575
tcp46      0      0  *.7575                 *.*                    LISTEN      131072 131072  49013      0
tcp4       0      0  127.0.0.1.7575         127.0.0.1.60685        TIME_WAIT   407296 146988  49013      0
tcp6       0      0  ::1.7575               ::1.60699              TIME_WAIT   407284 146808  49013      0
tcp6       0      0  ::1.7575               ::1.60700              TIME_WAIT   407284 146808  49013      0
obrienlabs-mbp15:_deployment michaelobrien$ netstat -vatn | grep 7688
tcp6       0      0  ::1.7688               ::1.60704              ESTABLISHED 406582 146808  49013      0
tcp6       0      0  ::1.60704              ::1.7688               ESTABLISHED 398196 146808  48165      0
tcp6       0      0  ::1.7688               ::1.60702              ESTABLISHED 406570 146808  49013      0
tcp6       0      0  ::1.60702              ::1.7688               ESTABLISHED 398185 146808  48165      0
tcp6       0      0  ::1.7688               ::1.60701              ESTABLISHED 407255 146808  49013      0
tcp6       0      0  ::1.60701              ::1.7688               ESTABLISHED 407628 146808  48165      0
tcp46      0      0  *.7688                 *.*                    LISTEN      131072 131072  49013      0
obrienlabs-mbp15:_deployment michaelobrien$ netstat -vatn | grep 8080
tcp4       0      0  127.0.0.1.8080         127.0.0.1.60584        FIN_WAIT_2  408104 146988  49013      0
tcp4     994      0  127.0.0.1.60584        127.0.0.1.8080         CLOSE_WAIT  408128 146988  42992      0
tcp46      0      0  *.8080                 *.*                    LISTEN      131072 131072  49013      0
udp4       0      0  *.*                    *.*                                196724   9216  38080      0

This code however is not integrated yet with our HA listener overrides on the constructor tree we bootstrap in Spring - the following scrubbed/distilled mirror POC of our corporate production code will be adjusted in the next post.

public class ExtendedHighlyAvailableGraphDatabaseFactory extends HighlyAvailableGraphDatabaseFactory {
    private HaMonitor haMonitor;
    private GraphDatabaseSettings.BoltConnector bolt = GraphDatabaseSettings.boltConnector( "0" );
    @Override
     protected GraphDatabaseBuilder.DatabaseCreator createDatabaseCreator(
                final File storeDir, final GraphDatabaseFactoryState state)  {
            return new GraphDatabaseBuilder.DatabaseCreator() {
                @Override
                public GraphDatabaseService newDatabase( final Map<String, String> config ) {
                    HighlyAvailableGraphDatabase haDB = 
                            new HighlyAvailableGraphDatabase( storeDir, config, state.databaseDependencies());
                    haMonitor.setDb(haDB);
                    HighAvailabilityMemberStateMachine memberStateMachine = 
                            haDB.getDependencyResolver().resolveDependency(HighAvailabilityMemberStateMachine.class);
                    if ( memberStateMachine != null ) {
                        memberStateMachine.addHighAvailabilityMemberListener(haMonitor);
                    }
                    return haDB;
                } };}
obrienlabs commented 7 years ago

We can get an HA embedded mode server up wrapped by a NeoServer - working out passing parameters with TypeSafe again still.

Dec 20, 2016 11:23:51 PM org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor <init>
2016-12-21 04:24:34.172+0000 INFO  Starting...
2016-12-21 04:25:02.323+0000 INFO  Write transactions to database disabled
2016-12-21 04:25:05.196+0000 INFO  Bolt enabled on 0.0.0.0:7688.
2016-12-21 04:25:05.314+0000 INFO  Initiating metrics...
2016-12-21 04:25:14.874+0000 INFO  Attempting to join cluster of [127.0.0.1:5001]
2016-12-21 04:25:16.883+0000 INFO  Could not join cluster of [127.0.0.1:5001]
2016-12-21 04:25:16.884+0000 INFO  Creating new cluster with name [neo4j.ha]...
2016-12-21 04:25:16.923+0000 INFO  Instance 1 (this server)  entered the cluster
2016-12-21 04:25:16.966+0000 INFO  Instance 1 (this server)  was elected as coordinator
2016-12-21 04:25:17.154+0000 INFO  I am 1, moving to master
2016-12-21 04:25:17.281+0000 INFO  Instance 1 (this server)  was elected as coordinator
2016-12-21 04:25:17.452+0000 INFO  I am 1, successfully moved to master
2016-12-21 04:25:17.506+0000 INFO  Instance 1 (this server)  is available as master at ha://127.0.0.1:6001?serverId=1 with StoreId{creationTime=1482199697648, randomId=7800059877674392627, storeVersion=15531981201765894, upgradeTime=1482199697648, upgradeId=1}
2016-12-21 04:25:25.306+0000 INFO  Database available for write transactions
2016-12-21 04:25:39.442+0000 INFO  Started.
2016-12-21 04:25:42.429+0000 INFO  Mounted REST API at: /db/manage
2016-12-21 04:26:02.890+0000 INFO  Remote interface available at http://localhost:7575/

haDB HighlyAvailableGraphDatabase (id=178)

obriensystems commented 7 years ago

example tomcat 8 war maven project for above POC https://github.com/obrienlabs/nbi-neo4j-embedded-aws-war in https://github.com/obrienlabs/nbi-neo4j-embedded-aws-war/issues/1

Endpoints

Add a node to the embedded graph db http://neo4j.ca-central-1.elasticbeanstalk.com/FrontController?action=graph (bolt enabled) http browser - user:neo4j pass:password http://neo4j.ca-central-1.elasticbeanstalk.com:7575/browser/

obriensystems commented 7 years ago

linking related (for the NPE) https://github.com/neo4j/neo4j/issues/8548

obriensystems commented 7 years ago

Note: we need to run with http enabled at all times using this workaround or we get

from
        <entry key="dbms.connector.http.enabled" value="true" />
        <entry key="dbms.connector.bolt.enabled" value="true" />
to
        <entry key="dbms.connector.http.enabled" value="false" />
        <entry key="dbms.connector.bolt.enabled" value="true" />
2017-01-13 21:46:21.401+0000 ERROR Failed to start Neo4j on unknown address. An HTTP connector must be configured to run the server
java.lang.IllegalArgumentException: An HTTP connector must be configured to run the server
    at org.neo4j.server.AbstractNeoServer.lambda$new$0(AbstractNeoServer.java:158)
    at java.util.Optional.orElseThrow(Optional.java:290)
    at org.neo4j.server.AbstractNeoServer.<init>(AbstractNeoServer.java:157)
    at org.neo4j.server.CommunityNeoServer.<init>(CommunityNeoServer.java:69)
    at org.neo4j.server.enterprise.EnterpriseNeoServer.<init>(EnterpriseNeoServer.java:105)
    at org.neo4j.server.enterprise.EnterpriseBootstrapper.createNeoServer(EnterpriseBootstrapper.java:45)
    at org.neo4j.server.ServerBootstrapper.start(ServerBootstrapper.java:90)
    at org.obrienlabs.nbi.graph.service.ExtendedHighlyAvailableGraphDatabaseFactory$1.newDatabase(ExtendedHighlyAvailableGraphDatabaseFactory.java:41)
obriensystems commented 7 years ago

blogspot http://neo4jogm.blogspot.ca/

oskarhane commented 6 years ago

You can now use Neo4j Browser that comes with Neo4j Desktop to connect to remote graphs directly via bolt. No need to enable http.

archenroot commented 5 years ago

@oskarhane - it doesn't work for me. I have in idea neo4j started in debug mode:

Connected to the target VM, address: '127.0.0.1:33525', transport: 'socket'
Dec 16, 2018 1:57:21 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:43 PM'
Dec 16, 2018 1:57:21 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:43 PM'
Dec 16, 2018 1:57:21 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:43 PM'
Dec 16, 2018 1:57:22 PM org.neo4j.driver.internal.logging.JULogger info
INFO: Direct driver instance 1922498059 created for server address 127.0.0.1:37087
2018-12-16 13:57:22,315 (main) INFO  a.p.g.i.s.s.n.Neo4jDriverHelper:32 - Created new driver instance for url=bolt://127.0.0.1:37087 in 356

But when trying to do :server connect and copy into Connect URL full string "bolt://127.0.0.1:37087" or without protocol specification just "127.0.0.1:37087", none works: ServiceUnavailable: Failed to establish connection in 5000ms

On the other hand when I try to do simply telnet:

zangetsu@venus: pts/0: 8 files 72Kb -> telnet 127.0.0.1 37087
Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.

Will try also neo4j shell... but what you claim to be working doesn't...

obriensystems commented 5 years ago

response_container_BBPPID{font-family: initial; font-size:initial; color: initial;} Worked at the time for that specific version - see caps and tests.  Either the code has evolved or your config is wrong /michael From: notifications@github.comSent: December 16, 2018 05:03To: neo4j-browser@noreply.github.comReply to: reply@reply.github.comCc: michael@obrienlabs.org; comment@noreply.github.comSubject: Re: [neo4j/neo4j-browser] Browser 7474 not available in Neo4j 3.1.0 embedded (#342) @oskarhane - it doesn't work for me. I have in idea neo4j started in debug mode:

Connected to the target VM, address: '127.0.0.1:33525', transport: 'socket' Dec 16, 2018 1:57:21 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:43 PM' Dec 16, 2018 1:57:21 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:43 PM' Dec 16, 2018 1:57:21 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:43 PM' Dec 16, 2018 1:57:22 PM org.neo4j.driver.internal.logging.JULogger info INFO: Direct driver instance 1922498059 created for server address 127.0.0.1:37087 2018-12-16 13:57:22,315 (main) INFO  a.p.g.i.s.s.n.Neo4jDriverHelper:32 - Created new driver instance for url=bolt://127.0.0.1:37087 in 356

But when trying to do :server connect and copy into Connect URL full string "bolt://127.0.0.1:37087" or without protocol specification just "127.0.0.1:37087", none works: ServiceUnavailable: Failed to establish connection in 5000ms On the other hand when I try to do simply telnet: zangetsu@venus: pts/0: 8 files 72Kb -> telnet 127.0.0.1 37087 Trying 127.0.0.1... Connected to 127.0.0.1. Escape character is '^]'.

Will try also neo4j shell... but what you claim to be working doesn't...

—You are receiving this because you commented.Reply to this email directly, view it on GitHub, or mute the thread.

jiaenqiu commented 2 years ago

Neo4j-v4.4.4,docker deploy 2022-03-08 05:51:28.365+0000 ERROR Client triggered an unexpected error [Neo.DatabaseError.General.UnknownError]: Lexical error at line 1, column 53. Encountered: after : "", reference 1a93f5f3-cd69-45a6-8232-32a5680aaca9.