search

neo4j-contrib / neo4j-etl

Data import from relational databases to Neo4j.
https://neo4j.com/developer/neo4j-etl/
Other
214 stars 46 forks source link

Cannot import data using neo4j-etl export #69

Open AnoYuan opened 4 years ago

AnoYuan commented 4 years ago

My process is like this： 1.generate mapping.json bash ./neo4j-etl generate-metadata-mapping --rdbms:url jdbc:mysql://mysql-ip:mysql-port --rdbms:user root --rdbms:password root --output-mapping-file "/var/lib/neo4j/mapping/test.json" --rdbms:schema test 2.Import data bash ./neo4j-etl export --rdbms:url jdbc:mysql://mysql-ip:mysql-port/test --rdbms:user root --rdbms:password root --destination /var/lib/neo4j/data/databases/test.db --import-tool /var/lib/neo4j/bin/ --csv-directory /var/lib/neo4j/import --neo4j:url bolt://neo4j-ip:7687 --neo4j:user neo4j --neo4j:password 123456 --options-file /var/lib/neo4j/conf/options.json --force --mapping-file "/var/lib/neo4j/mapping/test.json" --debug

logs: INFO: Reading options from file /root/docker_data/neo4j/conf/options.json. FINE: Connecting to database... FINE: Connected to database INFO: Reading metadata mapping from file: /root/docker_data/neo4j/mapping/test.json FINE: Using java.util.logging as the default logging framework FINE: -Dio.netty.eventLoopThreads: 16 FINE: -Dio.netty.threadLocalMap.stringBuilder.initialSize: 1024 FINE: -Dio.netty.threadLocalMap.stringBuilder.maxSize: 4096 FINE: -Dio.netty.noKeySetOptimization: false FINE: -Dio.netty.selectorAutoRebuildThreshold: 512 FINE: -Dio.netty.noUnsafe: false FINE: Java version: 8 FINE: sun.misc.Unsafe.theUnsafe: available FINE: sun.misc.Unsafe.copyMemory: available FINE: java.nio.Buffer.address: available FINE: direct buffer constructor: available FINE: java.nio.Bits.unaligned: available, true FINE: jdk.internal.misc.Unsafe.allocateUninitializedArray(int): unavailable prior to Java9 FINE: java.nio.DirectByteBuffer.(long, int): available FINE: sun.misc.Unsafe: available FINE: -Dio.netty.tmpdir: /tmp (java.io.tmpdir) FINE: -Dio.netty.bitMode: 64 (sun.arch.data.model) FINE: -Dio.netty.maxDirectMemory: 7495745536 bytes FINE: -Dio.netty.uninitializedArrayAllocationThreshold: -1 FINE: java.nio.ByteBuffer.cleaner(): available FINE: -Dio.netty.noPreferDirect: false FINE: org.jctools-core.MpscChunkedArrayQueue: available FINEST: instrumented a special java.util.Set into: sun.nio.ch.EPollSelectorImpl@7bba5817 FINEST: instrumented a special java.util.Set into: sun.nio.ch.EPollSelectorImpl@350aac89 FINEST: instrumented a special java.util.Set into: sun.nio.ch.EPollSelectorImpl@1c5920df FINEST: instrumented a special java.util.Set into: sun.nio.ch.EPollSelectorImpl@17f9d882 FINEST: instrumented a special java.util.Set into: sun.nio.ch.EPollSelectorImpl@79e4c792 FINEST: instrumented a special java.util.Set into: sun.nio.ch.EPollSelectorImpl@196a42c3 FINEST: instrumented a special java.util.Set into: sun.nio.ch.EPollSelectorImpl@4c60d6e9 FINEST: instrumented a special java.util.Set into: sun.nio.ch.EPollSelectorImpl@15043a2f FINEST: instrumented a special java.util.Set into: sun.nio.ch.EPollSelectorImpl@4a83a74a FINEST: instrumented a special java.util.Set into: sun.nio.ch.EPollSelectorImpl@1349883 FINEST: instrumented a special java.util.Set into: sun.nio.ch.EPollSelectorImpl@4b29d1d2 FINEST: instrumented a special java.util.Set into: sun.nio.ch.EPollSelectorImpl@7f485fda FINEST: instrumented a special java.util.Set into: sun.nio.ch.EPollSelectorImpl@28261e8e FINEST: instrumented a special java.util.Set into: sun.nio.ch.EPollSelectorImpl@d737b89 FINEST: instrumented a special java.util.Set into: sun.nio.ch.EPollSelectorImpl@8519cb4 FINEST: instrumented a special java.util.Set into: sun.nio.ch.EPollSelectorImpl@35dab4eb INFO: Direct driver instance 1485089044 created for server address ip:port INFO: Running ETL on Neo4j 3.5.5 - COMMUNITY INFO: Exporting from RDBMS to CSV... INFO: CSV directory: /root/docker_data/neo4j/import/csv-012 FINE: Executing command 'chmod 0777 /root/docker_data/neo4j/import/csv-012' FINER: Command finished [Command: 'chmod 0777 /root/docker_data/neo4j/import/csv-012', CommandResult { ExitValue: 0, Stdout: '', Stderr: '', DurationMillis: 9 }] FINE: Connecting to database... FINE: Connected to database INFO: Writing CSV headers for node NODE_test.ip_95b7044d-0ef9-4c6c-92cf-cfb0e09c7d66 FINEST: SELECT test.ip.id AS id, test.ip.id AS id, test.ip.ip AS ip, "Ip" AS _NODE_LABEL_ FROM test.ip Export time: 0.03 (s) INFO: Creating Neo4j store from CSV... WARNING: Neo4j is running! You can run neo4j-import tool only if the database is offline INFO: /root/docker_data/neo4j/data/databases/test.db

WARNING: Neo4j is running! You can run neo4j-import tool only if the database is offline WARNING: Neo4j is running! You can run neo4j-import tool only if the database is offline WARNING: Neo4j is running! You can run neo4j-import tool only if the database is offline WARNING: Neo4j is running! You can run neo4j-import tool only if the database is offline

What should I do?

jexp commented 4 years ago

Can you stop your neo4j database before you start your ETL process? Or use the online import mode.