Auth problem with Neo4j in Docker container

jankowtf commented 4 years ago

Hi Colin,

first of: thanks you and your team so much for neo4r and all your other packages - highly appreciated!

I just wanted to take neo4r for a spin, but somehow I can't connect to the Neo4j server running in a docker container.

First try

Keeping the defaults for username (neo4j) and password (neo4j)

Docker:

docker run --rm -p 7474:7474 -p 7687:7687 -v $(pwd)/data:/data --name neo4j neo4j

# Directories in use:
# home:         /var/lib/neo4j
# config:       /var/lib/neo4j/conf
# logs:         /logs
# plugins:      /var/lib/neo4j/plugins
# import:       /var/lib/neo4j/import
# data:         /var/lib/neo4j/data
# certificates: /var/lib/neo4j/certificates
# run:          /var/lib/neo4j/run
# Starting Neo4j.
# 2020-04-06 12:20:07.333+0000 INFO  ======== Neo4j 4.0.3 ========
#   2020-04-06 12:20:07.339+0000 INFO  Starting...
# 2020-04-06 12:20:18.046+0000 INFO  Bolt enabled on 0.0.0.0:7687.
# 2020-04-06 12:20:18.046+0000 INFO  Started.
# 2020-04-06 12:20:18.897+0000 INFO  Remote interface available at http://0.0.0.0:7474/

Reprex:

library(neo4r)
library(httr)

# Check if Neo4j is running at all:
GET("http://localhost:7474")
#> Response [http://localhost:7474]
#>   Date: 2020-04-06 10:48
#>   Status: 200
#>   Content-Type: application/json;charset=utf-8
#>   Size: 212 B
#> {
#>   "bolt_routing" : "neo4j://0.0.0.0:7687",
#>   "transaction" : "http://localhost:7474/db/{databaseName}/tx",
#>   "bolt_direct" : "bolt://0.0.0.0:7687",
#>   "neo4j_version" : "4.0.3",
#>   "neo4j_edition" : "community"

# Connect:
con <- neo4j_api$new(
  url = "http://localhost:7474",
  user = "neo4j",
  password = "neo4j"
)

# Unsuccesful ping:
con$ping()
#> [1] 404

# Trying explicit reset:
con$reset_user("neo4j")
con$reset_password("neo4j")

# Unsuccesful ping:
con$ping()
#> [1] 404

# Trying password from `README`:
con$reset_user("neo4j")
con$reset_password("password")

# Unsuccesful ping:
con$ping()
#> [1] 401

Second try

Providing/setting explicit username/password via environment variable

Docker:

docker run --rm -p 7474:7474 -p 7687:7687 -v $(pwd)/data:/data --env NEO4J_AUTH=neo4j/12345 --name neo4j neo4j

# Changed password for user 'neo4j'.
# Directories in use:
# home:         /var/lib/neo4j
# config:       /var/lib/neo4j/conf
# logs:         /logs
# plugins:      /var/lib/neo4j/plugins
# import:       /var/lib/neo4j/import
# data:         /var/lib/neo4j/data
# certificates: /var/lib/neo4j/certificates
# run:          /var/lib/neo4j/run
# Starting Neo4j.
# 2020-04-06 10:58:00.583+0000 INFO  ======== Neo4j 4.0.3 ========
#   2020-04-06 10:58:00.589+0000 INFO  Starting...
# 2020-04-06 10:58:03.144+0000 INFO  Bolt enabled on 0.0.0.0:7687.
# 2020-04-06 10:58:03.144+0000 INFO  Started.
# 2020-04-06 10:58:03.973+0000 INFO  Remote interface available at http://0.0.0.0:7474/

Reprex:

library(neo4r)
library(httr)

# Check if Neo4j is running at all:
GET("http://localhost:7474")
#> Response [http://localhost:7474]
#>   Date: 2020-04-06 11:00
#>   Status: 200
#>   Content-Type: application/json;charset=utf-8
#>   Size: 212 B
#> {
#>   "bolt_routing" : "neo4j://0.0.0.0:7687",
#>   "transaction" : "http://localhost:7474/db/{databaseName}/tx",
#>   "bolt_direct" : "bolt://0.0.0.0:7687",
#>   "neo4j_version" : "4.0.3",
#>   "neo4j_edition" : "community"

# Connect:
con <- neo4j_api$new(
  url = "http://localhost:7474",
  user = "neo4j",
  password = "12345"
)

# Unsuccesful ping:
con$ping()
#> [1] 404

# Trying explicit reset:
con$reset_user("neo4j")
con$reset_password("12345")

# Unsuccesful ping:
con$ping()
#> [1] 404

# Trying password from `README`:
con$reset_user("neo4j")
con$reset_password("password")

# Unsuccesful ping:
con$ping()
#> [1] 401

Session info

sessionInfo()
#> R version 3.6.1 (2019-07-05)
#> Platform: x86_64-pc-linux-gnu (64-bit)
#> Running under: Pop!_OS 19.10
#> 
#> Matrix products: default
#> BLAS:   /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.8.0
#> LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.8.0
#> 
#> locale:
#>  [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
#>  [3] LC_TIME=de_DE.UTF-8        LC_COLLATE=en_US.UTF-8    
#>  [5] LC_MONETARY=de_DE.UTF-8    LC_MESSAGES=en_US.UTF-8   
#>  [7] LC_PAPER=de_DE.UTF-8       LC_NAME=C                 
#>  [9] LC_ADDRESS=C               LC_TELEPHONE=C            
#> [11] LC_MEASUREMENT=de_DE.UTF-8 LC_IDENTIFICATION=C       
#> 
#> attached base packages:
#> [1] stats     graphics  grDevices datasets  utils     methods   base     
#> 
#> loaded via a namespace (and not attached):
#>  [1] compiler_3.6.1  magrittr_1.5    htmltools_0.4.0 tools_3.6.1    
#>  [5] yaml_2.2.1      Rcpp_1.0.4      stringi_1.4.6   rmarkdown_2.1  
#>  [9] highr_0.8       knitr_1.28      stringr_1.4.0   xfun_0.12      
#> [13] digest_0.6.25   rlang_0.4.5     renv_0.9.3      evaluate_0.14

fBedecarrats commented 4 years ago

Hi! Have you tried connecting to http://localhost:7474 with your web browser and entering your user name and password on the UI there to check that they had been correctly set?

jankowtf commented 4 years ago

Hi!

I did yesterday and was able to connect via my browser (Firefox).

But when I check now (both in Firefox and Chrome with emptied cache and docker run --rm -p 7474:7474 -v $(pwd)/data:/data --env NEO4J_AUTH=neo4j/12345 --name neo4j neo4j), I see the following error:

SessionExpired: WebSocket connection failure. Due to security constraints in your web browser, the reason for the failure is not available to this Neo4j Driver. Please use your browsers development console to determine the root cause of the failure. Common reasons include the database being unavailable, using th

Strangely, I also get the error after stopping, removing the $(pwd)/data directory and restarting with docker run --rm -p 7474:7474 -v $(pwd)/data:/data --name neo4j neo4j again and using the defaults neo4j/neo4j

What am I doing wrong?

jankowtf commented 4 years ago

Ah, found the reason why I couldn't connect via the browser: yesterday while troubleshooting, I took out the second port forwarding -p 7687:7687 which as I understand is for Bolt.

When I bring it back, start the container with docker run --rm -p 7474:7474 -p 7687:7687 -v $(pwd)/data:/data --name neo4j neo4j and request http://localhost:7474 via my browser, the server asks me to specify a new password (I used 12345 again for testing) and lets me connect afterwards.

However, when I use that very password in con <- neo4j_api$new(), I still get a 404 (which is also strange; I'd expected a 401 rather than a 404 here)

fBedecarrats commented 4 years ago

Hi @rappster, I think the problem comes from the fact that with you're running the newest version of Neo4J (version 4), while neo4r only works with Neo4J version 3. If you run the same command, adding ":3.5.3" at the end, then it works. In terminal:
docker run --rm -p 7474:7474 -p 7687:7687 -v $(pwd)/data:/data --env NEO4J_AUTH=neo4j/12345 --name neo4j neo4j:3.5.3 Then in R:

> library(neo4r)
> con <- neo4j_api$new(
+   url = "http://localhost:7474", 
+   user = "neo4j", 
+   password = "12345"
+ )
> con$ping()
[1] 200
>

jankowtf commented 4 years ago

@fBedecarrats thanks a lot, works like a charm with 3.5.3.

Are you planning on making neo4j work with 4.x versions anytime soon or will that still be a while?

ColinFay commented 4 years ago

As far as I know it's only the ping() method that fails, other methods still work. Can you confirm?

jankowtf commented 4 years ago

@ColinFay yeb, I can confirm that.

Just checked again with the latest Neo4j version and a CREATE statement went through just fine.

Thanks for looking into this

earnaud commented 3 years ago

Hi !

I am having an issue falling under the scope of the name of this issue.

( EDIT: this might also suit in #64 )

I create a shiny container on a side, and a neo4j container one one other side. Launching both containers is ok, connecting to both the shiny app and neo4j browser from the host is ok, but having the shiny app connecting to the neo4j DB is not working properly. I made it work in local.

I would have expected my shiny app to connect by the docker network. I set up the connection as follow:

con = neo4r::neo4j_api$new(
        url = "http://0.0.0.0:7474", # 0.0.0.0 is better on my os aka ubuntu
        user = "neo4j",
        password = "pw"
      )

but get this error:

> con
<neo4j connection object>
Error: Failed to connect to 0.0.0.0 port 7474: Connection refused
No registered Connection
(Wrong credentials or hostname)

However, using the same id/pw in the neo4j browser results in a valid connection.

I am not proficient with docker, so I might have forgotten something. Please find attached my docker-compose file.

docker-compose.yml

version: '3'
services:
  neo4j:
    image: neo4j:3.5-enterprise
    restart: unless-stopped
    container_name: neo4j
    depends_on: ["shiny4cartosib"]
    ports:
      - 7474:7474 # HTTP 
      - 7473:7473 # HTTPS
      - 7687:7687 # bolt
    volumes:
      # import data
      - ./neo4j_setup/import/:/var/lib/neo4j/import
      - ./neo4j_setup/:/neo4j_setup
    environment:
      - NEO4J_ACCEPT_LICENSE_AGREEMENT=yes
      # Install APOC library
      - NEO4JLABS_PLUGINS='[\"apoc\"]'
      # Raise memory limits
      - NEO4J_dbms_memory_pagecache_size=1G
      - NEO4J_dbms.memory.heap.initial_size=1G
      - NEO4J_dbms_memory_heap_max__size=1G
      # Authentication setup
      - NEO4J_AUTH=neo4j/pw
  shiny4cartosib:
    build: .
    # image: shiny4cartosib:local
    container_name: shiny4cartosib
    ports:
      - 3838:3838

earnaud commented 3 years ago

My problem is solved (but sorry, could not show the whole repo since it is private)

neo4j-rstats / neo4r