search

modmore / importX

Addon for MODX Revolution which will take your CSV Formatted files (or raw paste) and transforms it into resources. Funded by Working Party (digital agency based in Sydney, @workingparty)
http://www.markhamstra.com/
23 stars 17 forks source link

Import fails if last field is empty #54

Open apathyuk opened 10 years ago

apathyuk commented 10 years ago

hello

Firstly ImportX saved me from having to manually add loads of data so the pain was worth it but what I found is if the last field can be blank in some records, then errors occur.

My hack solution was to move a field that always had a value to the end.

James

--- Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/2504550-import-fails-if-last-field-is-empty?utm_campaign=plugin&utm_content=tracker%2F1511225&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F1511225&utm_medium=issues&utm_source=github).
Mark-H commented 10 years ago

Hmmm I guess it probably complained about an incorrect number of fields in the row?

Right now the CSV parser is something custom built, but I am thinking of using something more robust that supports more CSV features in the future that should sort out stuff like this..

apathyuk commented 10 years ago

Yes, it complained about the number of fields.

I needed to import data compiled by a 3rd party in an Excel file and these are the steps (not including moving a field to the end of the table):

1) Choose a deliminator and search the Excel file to make sure it is not being used. I used pipe in this case. 2) Choose a temp comma replacement and search the Excel file to make sure it is not being used. I used hash in this case. 3) Find for all double quotes and remove them from all cells 4) Find all carriage returns within cells and replace with comma space (otherwise they appear on a new line in the CSV) 5) Replace all commas with the temp comma replacement (#) 6) Export to CSV 7) Open in text edior 8) Remove all double quotes (as otherwise ImportX will import them) 9) Replace all commas with the new deliminator (I used pipe) 10) Switch back the comma replacement with commas 11) Remove the blank lines at the bottom

Its still far less effort than having to manually enter the data and deal with the data capture errors but it would be great if one could import an Excel file directly as this would remove the majority of the steps.

Kind regards

James