search

neo4j-contrib / neo4j-etl

Data import from relational databases to Neo4j.
https://neo4j.com/developer/neo4j-etl/
Other
214 stars 46 forks source link

Get error when importing data from Oracle 12c in Docker Container. #1

Closed rohankharwar closed 6 years ago

rohankharwar commented 7 years ago

I am using neo4j-etl tool to import data from Oracle 12c into Neo4j. Oracle is setup on Docker. However when I run neo4j-etl I get the following error. 

java.lang.IllegalArgumentException: No enum constant org.neo4j.etl.sql.metadata.SqlDataType.BINARY_DOUBLE
    at java.lang.Enum.valueOf(Enum.java:238)
    at org.neo4j.etl.sql.metadata.SqlDataType.valueOf(SqlDataType.java:7)
    at org.neo4j.etl.sql.metadata.SqlDataType.parse(SqlDataType.java:69)
    at org.neo4j.etl.sql.metadata.TableInfoAssembler.lambda$createColumnsMap$3(TableInfoAssembler.java:66)
    at org.neo4j.etl.sql.metadata.TableInfoAssembler$$Lambda$10/1403704789.apply(Unknown Source)
    at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
    at java.util.HashMap$EntrySpliterator.forEachRemaining(HashMap.java:1683)
    at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:512)
    at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:502)
    at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
    at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
    at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
    at org.neo4j.etl.sql.metadata.TableInfoAssembler.createColumnsMap(TableInfoAssembler.java:70)
    at org.neo4j.etl.sql.metadata.TableInfoAssembler.createTableInfo(TableInfoAssembler.java:43)
    at org.neo4j.etl.commands.DatabaseInspector.buildSchema(DatabaseInspector.java:51)
    at org.neo4j.etl.commands.DatabaseInspector.buildSchemaExport(DatabaseInspector.java:39)
    at org.neo4j.etl.commands.rdbms.GenerateMetadataMapping.call(GenerateMetadataMapping.java:92)
    at org.neo4j.etl.commands.rdbms.GenerateMetadataMapping.call(GenerateMetadataMapping.java:28)
    at org.neo4j.etl.cli.rdbms.ExportFromRdbmsCli.createMetadataMappings(ExportFromRdbmsCli.java:260)
    at org.neo4j.etl.cli.rdbms.ExportFromRdbmsCli.run(ExportFromRdbmsCli.java:219)
    at org.neo4j.etl.util.CliRunner.run(CliRunner.java:42)
    at org.neo4j.etl.util.CliRunner.run(CliRunner.java:35)
    at org.neo4j.etl.NeoIntegrationCli.main(NeoIntegrationCli.java:44)

Here is my command

Rohans-MacBook-Pro:neo4j-etl-cli-1.1.0 rohankharwar$ ./bin/neo4j-etl oracle export --url jdbc:oracle:thin:H000/password@localhost:1521/xe --schema H000 --user H000 --password password --import-tool /Users/rohankharwar/Documents/neo4j-enterprise-3.2.2/bin  --options-file /Users/rohankharwar/tmp/nokia/options.json --csv-directory /Users/rohankharwar/tmp/nokia --destination /Users/rohankharwar/Documents/neo4j-enterprise-3.2.2/data/databases/nokiacomptel.db/ --quote '"' --force
- Skipping reading import options from file because file [/Users/rohankharwar/tmp/nokia/options.json] doesn't exist.
Creating RDBMS to CSV mappings...
Command failed due to error (IllegalArgumentException: No enum constant org.neo4j.etl.sql.metadata.SqlDataType.BINARY_DOUBLE). Rerun with --debug flag for detailed diagnostic information.
jexp commented 7 years ago

Can we fix this for the particular case but also for the general case where we don't have a correct mapping?

I.e. mapping numbers to long/double and everything else to string?

jexp commented 7 years ago

We would need the fix by early next week.

inserpio commented 7 years ago

As we still don't manage arrays, the candidate rdbms to neo4j datatype mapping could be:

        switch (javaSqlTypeGroup) {
            case bit:
                return Neo4jDataType.Byte;
            case integer:
                return Neo4jDataType.Long;
            case real:
                return Neo4jDataType.Double;
            case object:
            case large_object:
            case reference:
            case unknown:
            case binary:
                return null;
            case id:
            case url:
            case xml:
            case temporal:
            case character:
            default:
                return Neo4jDataType.String;
        }

BLOB, CLOB are treated as large_object by the schemacrawler (see here) and we esclude them from the import (by returning null (in the above switch), otherwise the risk is having a node property as big as the content of the B/CLOB.

If ok, I can release a patch for that.

jexp commented 6 years ago

This should be fixed for some months now @rohankharwar

could you re-check?