Export-binary doesn't work correctly

[kayrus]

Steps to reproduce...

• Run neo4j with the working data directory.

export-binary -o /tmp/export.bin
Wrote to binary file /tmp/export.bin 0. 100%: nodes = 1646 rels = 2271 properties = 7884 time 321 ms total 321 ms

• Then stop neo4j, clean-up data directory and run import:

import-binary -i /tmp/export.bin
Binary import file /tmp/export.bin rel-type RELATED_TO batch-size 10000 use disk-cache false
Importing Indices and Constraints
Index Import Complete
Importing Nodes and Edges
finish after 1243 row(s)  0. 100%: nodes = 1228 rels = 0 properties = 3874 time 2578 ms total 2578 ms
ConstraintViolationException: Node 688 already exists with label User and property "id"=[30]

I use the following query on working data:

$ curl -H 'Accept: application/json; charset=UTF-8' -H 'Content-Type: application/json'-X POST http://localhost:7474/db/data/cypher -d'{ "query" : "match n where ID(n) = 688 return n", "params": {} }'
{
  "columns" : [ "n" ],
  "data" : [ ]

And the same query on restored database:

$ curl -H 'Accept: application/json; charset=UTF-8' -H 'Content-Type: application/json' -X POST http://localhost:7474/db/data/cypher -d'{ "query" : "match n where ID(n) = 688 return n", "params": {} }'
{
  "columns" : [ "n" ],
  "data" : [ [ {
    "metadata" : {
      "id" : 688,
      "labels" : [ "User" ]
    },
    "data" : {
%data was here%
    },
    "paged_traverse" : "http://localhost:7474/db/data/node/688/paged/traverse/{returnType}{?pageSize,leaseTime}",
    "outgoing_relationships" : "http://localhost:7474/db/data/node/688/relationships/out",
    "outgoing_typed_relationships" : "http://localhost:7474/db/data/node/688/relationships/out/{-list|&|types}",
    "create_relationship" : "http://localhost:7474/db/data/node/688/relationships",
    "labels" : "http://localhost:7474/db/data/node/688/labels",
    "traverse" : "http://localhost:7474/db/data/node/688/traverse/{returnType}",
    "extensions" : { },
    "all_relationships" : "http://localhost:7474/db/data/node/688/relationships/all",
    "all_typed_relationships" : "http://localhost:7474/db/data/node/688/relationships/all/{-list|&|types}",
    "property" : "http://localhost:7474/db/data/node/688/properties/{key}",
    "self" : "http://localhost:7474/db/data/node/688",
    "incoming_relationships" : "http://localhost:7474/db/data/node/688/relationships/in",
    "properties" : "http://localhost:7474/db/data/node/688/properties",
    "incoming_typed_relationships" : "http://localhost:7474/db/data/node/688/relationships/in/{-list|&|types}"
  } ] ]
}

[jexp]

Would it perhaps possible to share the database you exported with me? To test it out?

Note that node-id's can't be restored they are incrementing in the database.

Michael

Am 05.09.2016 um 17:40 schrieb kayrus notifications@github.com:

Steps to reproduce...

Run neo4j with the working data directory. export-binary -o /tmp/export.bin Wrote to binary file /tmp/export.bin 0. 100%: nodes = 1646 rels = 2271 properties = 7884 time 321 ms total 321 ms Then stop neo4j, clean-up data directory and run import: import-binary -i /tmp/export.bin Binary import file /tmp/export.bin rel-type RELATED_TO batch-size 10000 use disk-cache false Importing Indices and Constraints Index Import Complete Importing Nodes and Edges finish after 1243 row(s) 0. 100%: nodes = 1228 rels = 0 properties = 3874 time 2578 ms total 2578 ms ConstraintViolationException: Node 688 already exists with label User and property "id"=[30] I use the following query on working data:

$ curl -H 'Accept: application/json; charset=UTF-8' -H 'Content-Type: application/json'-X POST http://localhost:7474/db/data/cypher -d'{ "query" : "match n where ID(n) = 688 return n", "params": {} }' { "columns" : [ "n" ], "data" : [ ] And the same query on restored database:

$ curl -H 'Accept: application/json; charset=UTF-8' -H 'Content-Type: application/json' -X POST http://localhost:7474/db/data/cypher -d'{ "query" : "match n where ID(n) = 688 return n", "params": {} }' { "columns" : [ "n" ], "data" : [ [ { "metadata" : { "id" : 688, "labels" : [ "User" ] }, "data" : { %data was here% }, "paged_traverse" : "http://localhost:7474/db/data/node/688/paged/traverse/{returnType}{?pageSize,leaseTime}", "outgoing_relationships" : "http://localhost:7474/db/data/node/688/relationships/out", "outgoing_typed_relationships" : "http://localhost:7474/db/data/node/688/relationships/out/{-list|&|types}", "create_relationship" : "http://localhost:7474/db/data/node/688/relationships", "labels" : "http://localhost:7474/db/data/node/688/labels", "traverse" : "http://localhost:7474/db/data/node/688/traverse/{returnType}", "extensions" : { }, "all_relationships" : "http://localhost:7474/db/data/node/688/relationships/all", "all_typed_relationships" : "http://localhost:7474/db/data/node/688/relationships/all/{-list|&|types}", "property" : "http://localhost:7474/db/data/node/688/properties/{key}", "self" : "http://localhost:7474/db/data/node/688", "incoming_relationships" : "http://localhost:7474/db/data/node/688/relationships/in", "properties" : "http://localhost:7474/db/data/node/688/properties", "incoming_typed_relationships" : "http://localhost:7474/db/data/node/688/relationships/in/{-list|&|types}" } ] ] } — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/jexp/neo4j-shell-tools/issues/98, or mute the thread https://github.com/notifications/unsubscribe-auth/AAEHY9poVm4IuTHc3_ILA_EDawVNFHGeks5qnDfUgaJpZM4J1LG2.

[kayrus]

@jexp unfortunately I can not provide this db. but i just wonder is there a workaround? The only requirement: I don't have to specify DB schemes every time I make a dump.

[kayrus]

Maybe detailed error message will help:

finish after 1243 row(s)  0. 100%: nodes = 1228 rels = 0 properties = 3874 time 2419 ms total 2419 ms
ERROR (-v for expanded information):
        ConstraintViolationException: Node 688 already exists with label User and property "id"=[30]
org.neo4j.graphdb.ConstraintViolationException: Node 688 already exists with label User and property "id"=[30]
        at org.neo4j.kernel.impl.core.NodeProxy.setProperty(NodeProxy.java:297)
        at org.neo4j.shell.tools.imp.format.kryo.KryoReader.readBinaryDump(KryoReader.java:96)
        at org.neo4j.shell.tools.imp.ImportBinaryApp.readKryo(ImportBinaryApp.java:86)
        at org.neo4j.shell.tools.imp.ImportBinaryApp.execute(ImportBinaryApp.java:51)
        at org.neo4j.shell.impl.AbstractAppServer.interpretLine(AbstractAppServer.java:126)
        at org.neo4j.shell.kernel.GraphDatabaseShellServer.interpretLine(GraphDatabaseShellServer.java:103)
        at org.neo4j.shell.impl.AbstractClient.evaluate(AbstractClient.java:149)
        at org.neo4j.shell.impl.AbstractClient.evaluate(AbstractClient.java:133)
        at org.neo4j.shell.StartClient.grabPromptOrJustExecuteCommand(StartClient.java:380)
        at org.neo4j.shell.StartClient.tryStartLocalServerAndClient(StartClient.java:305)
        at org.neo4j.shell.StartClient.startLocal(StartClient.java:277)
        at org.neo4j.shell.StartClient.start(StartClient.java:210)
        at org.neo4j.shell.StartClient.main(StartClient.java:145)
Caused by: org.neo4j.kernel.api.exceptions.schema.UniquePropertyConstraintViolationKernelException: Node 688 already exists with label 0 and property 1=30
        at org.neo4j.kernel.impl.api.ConstraintEnforcingEntityOperations.validateNoExistingNodeWithLabelAndProperty(ConstraintEnforcingEntityOperations.java:165)
        at org.neo4j.kernel.impl.api.ConstraintEnforcingEntityOperations.nodeSetProperty(ConstraintEnforcingEntityOperations.java:140)
        at org.neo4j.kernel.impl.api.LockingStatementOperations.nodeSetProperty(LockingStatementOperations.java:479)
        at org.neo4j.kernel.impl.api.OperationsFacade.nodeSetProperty(OperationsFacade.java:922)
        at org.neo4j.kernel.impl.core.NodeProxy.setProperty(NodeProxy.java:293)
        ... 12 more

[kayrus]

@jexp any clue on what could be wrong? import ordering or something else?

[kayrus]

@jexp I've analyzed neo4j, it appeared that one user entry was incorrect. the original db contained two users with the id=30. I have no idea how it could happen. I removed incorrect user, made a backup, then restored. Everything went fine.

# get the ID of the incorrect user
MATCH (n) WHERE n.id=30 and n.field='bla' RETURN ID(n);
# delete relations
START n=node(12554) MATCH (n)-[r]->() DELETE r;
# delete nodeid
START n=node(12554) DELETE n;