search

jexp / batch-import

generic csv file neo4j batch importer
https://neo4j.com/docs/operations-manual/current/tools/import/
385 stars 157 forks source link

Issue when importing UTF-8 csv file #54

Open supersen opened 11 years ago

supersen commented 11 years ago

When a UTF-8 csv file is imported and property type is specified the importer throws dataype not found ex. for header in the file "name:string:users" the importer throws

Exception in thread "main" java.lang.IllegalArgumentException: Unknown Type string at org.neo4j.batchimport.importer.Type.fromString(Type.java:164) at org.neo4j.batchimport.importer.AbstractLineData.createHeaders(AbstractLineData.java:44) at org.neo4j.batchimport.importer.CsvLineData.(CsvLineData.java:16) at org.neo4j.batchimport.Importer.createLineData(Importer.java:144) at org.neo4j.batchimport.Importer.importNodes(Importer.java:84) at org.neo4j.batchimport.Importer.doImport(Importer.java:199) at org.neo4j.batchimport.Importer.main(Importer.java:74)

twish commented 10 years ago

I get the same when using :label but it then tells med "Unknown Type label"

Also using UTF-8 csv files.

jexp commented 10 years ago

What version?

Sent from mobile device

Am 19.12.2013 um 15:39 schrieb Johannes Tveitan notifications@github.com:

I get the same when using :label but it then tells med "Unknown Type label"

Also using UTF-8 csv files.

— Reply to this email directly or view it on GitHub.

twish commented 10 years ago

Oh, sorry. Using the downloadable zip from the 2.0 branch. Tried to adding labels to the import.

Regards Johannes

jexp commented 10 years ago

Can you share the header from your files?

Perhaps an extra space a the end?

Sent from mobile device

Am 19.12.2013 um 16:10 schrieb Johannes Tveitan notifications@github.com:

Oh, sorry. Using the downloadable zip from the 2.0 branch. Tried to adding labels to the import.

Regards Johannes

— Reply to this email directly or view it on GitHub.

twish commented 10 years ago

Absolutley, here comes the header in the file:

i:id    node_id last_name   first_name  initials    suffix  date_insterted  date_updated    type:label

The error looks like follows, just for reference:

Using Existing Configuration File

Total import time: 1 seconds 
Exception in thread "main" java.lang.IllegalArgumentException: Unknown Type label
    at org.neo4j.batchimport.importer.Type.fromString(Type.java:172)
    at org.neo4j.batchimport.importer.AbstractLineData.createHeaders(AbstractLineData.java:46)
    at org.neo4j.batchimport.importer.ChunkerLineData.<init>(ChunkerLineData.java:19)
    at org.neo4j.batchimport.Importer.createLineData(Importer.java:174)
    at org.neo4j.batchimport.Importer.importNodes(Importer.java:93)
    at org.neo4j.batchimport.Importer.doImport(Importer.java:228)
    at org.neo4j.batchimport.Importer.main(Importer.java:83)

I have double checked if there would be a space added at the end and since I generate the label header it's an easy check. Could not find any extra space or other suspect characters.

As I mentioned before the file is in UTF-8 and importer is runnig with:

batch_import.csv.quotes=false
twish commented 10 years ago

Hi again, was just checking in to see if there is any news on this front?

Regards

jexp commented 10 years ago

I think it is what has been reported in: https://github.com/jexp/batch-import/issues/74

Which was windows newlines at the end?

twish commented 10 years ago

It seems you are correct sir. I do indeed have windows eol signs (CRLF). Will try to convert the CSV files to linux file endings.

Sorry for the trouble, missed the reffered issue.

mrkale commented 10 years ago

I have imported CSV with Windows eol signt (CRLF) on desktop PC without any problem. I think it is not an issue.

Regards Libor Gabaj

On Wed, Jan 8, 2014 at 2:11 PM, Johannes Tveitan notifications@github.comwrote:

It seems you are correct sir. I do indeed have windows eol signs (CRLF). Will try to convert the CSV files to linux file endings.

Sorry for the trouble, missed the reffered issue.

— Reply to this email directly or view it on GitHubhttps://github.com/jexp/batch-import/issues/54#issuecomment-31829504 .

twish commented 10 years ago

I think the problem is when using the custom CSV reader when working with batch_import.csv.quotes=false. Then it has some problems.

I have now tried converting all my CSV files to linux eol and the import seems to be working. So i cross my fingers and hope that the rest will work aswell.

Thanks a bunch for the help!