neo4j-contrib / neo4j-apoc-procedures

Awesome Procedures On Cypher for Neo4j - codenamed "apoc"                     If you like it, please ★ above ⇧            
https://neo4j.com/labs/apoc
Apache License 2.0
1.7k stars 494 forks source link

couchbase connectivity #645

Closed graphadvantage closed 6 years ago

graphadvantage commented 6 years ago

Hi

I need some guidance on how to use apoc.couchbase with couchbase 4.6 and 5.

5 has deprecated bucket passwords in favor of user passwords 4.6 has bucket passwords using sasl auth

Please provide an example of proper connection syntax for each version (eg with bucket passwords for 4.6 and without bucket passwords for 5.0)

https://forums.couchbase.com/t/running-n1ql-via-rest-api-with-bucket-password/12084/2

I'd also like some guidance on proper versions of java-client, core-io and rx-java for both 4.6 and 5.0

thanks for your help!

Michael

inserpio commented 6 years ago

Hi Michael, I'll ping you back later for any clarification (dinner time for me now), but I haven't tested the apoc with couchbase 5, maybe some adjustment are required.

Lib versions are documented in the apoc guide

graphadvantage commented 6 years ago

Did some testing - found the following:

apoc.couchbase 3.1.3.7 and Neo4j 3.1.7 works with both Couchbase 4.6, 5.0 Couchbase 5.0 is backward compatible with the java-client 2.3.1 set of dependencies

apoc.couchbase 3.2.3.5 and Neo4j 3.2.7 does not work with either Couchbase 4.6, 5.0

apoc.couchbase 3.3.0.1 and Neo4j 3.3.0 does not work with either Couchbase 4.6, 5.0

note: Couchbase 5 uses the updated SDK, latest release is:

Version 2.5.2 (8 November 2017) Version 2.5.2 is the third release of the 2.5 series, bringing enhancements and bugfixes over the last stable release.

Download | API Reference | Core API Reference

The supported and tested dependencies for this release are:

com.couchbase.client:java-client:2.5.2 com.couchbase.client:core-io:1.5.2 io.reactivex:rxjava:1.3.3

graphadvantage commented 6 years ago

Hi @inserpio, @jexp

Were you able to confirm?

Thanks

Michael

inserpio commented 6 years ago

Yeah, that's correct. We're thinking about how to manage different apoc vs difference couchbase versions. We need to combine both... Anyway in the meantime I can provide you an updated version of apoc on the fly. Which neo4j version do you use?

graphadvantage commented 6 years ago

I'd love builds for 3.3.0 and 3.2.7 - thanks, Michael

inserpio commented 6 years ago

Hi @graphadvantage, could you try with the attached distribution? It contains an upgraded version of the couchbase procedures with their dependencies. Just unzip it into the plugin folder.

I introduced two new parameters username and password to all procedures, right after the bucket parameter.

For example (Couchbase 5.0): CALL apoc.couchbase.get(['localhost'], 'default', 'apoc', 'couchbase', 'artist:vincent_van_gogh')

where apoc is the user and couchbase the password granted for accessing to the default bucket.

If you want to skip the authentication phase (mostly for backward compatibility) then you just have to pass null for both username and password.

For example (Couchbase 4.6): CALL apoc.couchbase.get(['localhost'], null, null, 'couchbase', 'artist:vincent_van_gogh')

Waiting for your feedback! Thanks, Lorenzo

apoc-3.2.3.5-all.zip

graphadvantage commented 6 years ago

Hi @inserpio

Thanks Lorenzo, this is great! - if you wouldn't mind making a 3.3 build I'll test that as well (I use both versions).

Thanks,

Michael

graphadvantage commented 6 years ago

Tested on Couchbase 5 with 3.2.7 - works well, thank you.

using created user 'neo4j', password 'couchdb' with read/write privileges on beer-sample

CALL apoc.couchbase.query(['127.0.0.1'], 'beer-sample','neo4j','couchdb','SELECT * FROM `beer-sample` WHERE name = "Amendment Pale Ale"') yield queryResult
RETURN queryResult

{
  "beer-sample": {
    "abv": 5.2,
    "name": "Amendment Pale Ale",
    "description": "Rich golden hue color. Floral hop with sweet malt aroma. Medium mouth feel with malt sweetness, hop quenching flavor and well-balanced bitterness.",
    "upc": 0,
    "style": "American-Style Pale Ale",
    "brewery_id": "21st_amendment_brewery_cafe",
    "category": "North American Ale",
    "type": "beer",
    "ibu": 0,
    "updated": "2010-07-22 20:00:20",
    "srm": 0
  }
}
CALL apoc.couchbase.get(['127.0.0.1'], 'beer-sample','neo4j','couchdb', '21st_amendment_brewery_cafe-amendment_pale_ale') YIELD content
RETURN content

{
  "name": "Amendment Pale Ale",
  "style": "American-Style Pale Ale",
  "description": "Rich golden hue color. Floral hop with sweet malt aroma. Medium mouth feel with malt sweetness, hop quenching flavor and well-balanced bitterness.",
  "brewery_id": "21st_amendment_brewery_cafe",
  "srm": 0,
  "upc": 0,
  "abv": 5.2,
  "updated": "2010-07-22 20:00:20",
  "ibu": 0,
  "category": "North American Ale",
  "type": "beer"
}
CALL apoc.couchbase.exists(['127.0.0.1'], 'beer-sample','neo4j','couchdb', '21st_amendment_brewery_cafe-amendment_pale_ale')  YIELD value
RETURN value

true
WITH '{"beer-sample":{"abv":5.2,"name":"Amendment Pale Ale","description":"Rich golden hue color. Floral hop with sweet malt aroma. Medium mouth feel with malt sweetness, hop quenching flavor and well-balanced bitterness.","upc":0,"style":"American-Style Pale Ale","brewery_id":"21st_amendment_brewery_cafe","category":"North American Ale","type":"beer","ibu":0,"updated":"2010-07-22 20:00:20","srm":0}}' AS json
, "my-test-beer-doc" AS doc_id
CALL apoc.couchbase.insert(['127.0.0.1'], 'beer-sample','neo4j','couchdb', doc_id, json) yield id, expiry, cas, mutationToken, content
RETURN id, expiry, cas, mutationToken, content

"my-test-beer-doc" | 0 | 1511152538541686784 | null | {   "beer-sample": {     "name": "Amendment Pale Ale",     "style": "American-Style Pale Ale",     "description": "Rich golden hue color. Floral hop with sweet malt aroma. Medium mouth feel with malt sweetness, hop quenching flavor and well-balanced bitterness.",     "brewery_id": "21st_amendment_brewery_cafe",     "srm": 0,     "upc": 0,     "abv": 5.2,     "updated": "2010-07-22 20:00:20",     "ibu": 0,     "category": "North American Ale",     "type": "beer"   } }

Thanks! Michael

jexp commented 6 years ago

@graphadvantage how do the 3.2.x and 3.3.x fail with the couchbase 5.0 driver ? I'm not sure what our difference is between 3.1.x and the other versions that makes it fail.

jexp commented 6 years ago

@inserpio to capture different couchbase versions, if needed we can do:

  1. create an interface that contains the different operations we need
  2. have several implementations for the different couchbase drivers
  3. try to instantiate the newest variant and on failure try the next older version, until it matches the one that the user provided as jar or none matching is found then it should fail.
graphadvantage commented 6 years ago

@jexp - @inserpio 's changes for 3.2.7 worked, perhaps if you just pull them into a 3.3 branch I'm happy to test. The main difference is that CB deprecated bucket-level auth in 4 favor of more general RBAC model in 5. So now basic auth is required.

InverseFalcon commented 6 years ago

Hello, checking for a status on this one.

Also requesting clearer documentation of which versions / approaches are supported (with respect to passing credentials when making the calls), and corresponding examples.

We seem to reference this in the current APOC docs, would be good to know how this fits into what's currently supported, and how to use it:

apoc.couchbase.<key>.uri=couchbase-url-with-credentials

albertodelazzari commented 6 years ago

Hi, the idea (in order to have only on signature modification) is to define three properties:

To use the key we need to change the first parameter from a list of strings to a single string (as it now works for MongoDB connections "hostOrKey"). This is a breaking change.

I don't think it makes sense to put credentials into the URL because we can have more than one URL and the authentication is a cluster concept (from the client point of view) instead of a single machine one. It's better to have them separate from the URL.

And we also want to change the syntax of the "bucket" parameter in order to accept also passwords for buckets. For example, if you need to specify a password for a bucket you will define the bucketName parameter as "bucketName:bucketPassword". Couchbase 4.x supports passwords for buckets while for Couchbase 5.x the access control is user based and there is no "bucket's password" concept as the previous version.

We planned to support Couchbase 4.x and 5.x versions. The SDK version will be updated to 2.5 as reported here