Import CSV with double quotes error

[GoogleCodeExporter]

What steps will reproduce the problem?
1. Importing a file with the text below using 'Fields Seperated by: Comma'
and 'Fields enclosed by: Double quotes ("), if necessary (Standard)'

VAUXHALL,"Vectra, MY2006","1.8i 16v 5 Door Hatchback with 15""/16""/17""
tyre, wide ratio gear box and Z18XE
engine",M5,1796,Petrol,10.4,5.5,7.3,27.1,51.3,38.6,175,1272,IV,72.0,0.232,0.020,
0.006

What is the expected output? What do you see instead?

"wide ratio gearbox and Z18XE engine" is incorrectly imported into column
4. It should remain in column 3

What version of the product are you using? On what target Application (e.g.
Firefox 2.0.0.14, Sunbird 0.8, etc.)? On what operating system?
Extension Version = 0.3.4
Application = 3.0.1
OS = Windows XP

Original issue reported on code.google.com by djb...@gmail.com on 10 Aug 2008 at 6:05

[GoogleCodeExporter]

This issue has already been fixed in version 0.3.5
Please update and confirm.

Original comment by mrinal.k...@gmail.com on 10 Aug 2008 at 6:26

[GoogleCodeExporter]

Tried 0.3.5 with no success. It appears there is a regression. When using the 
same
.csv file with 0.3.4 it imports 1713 rows. Using 0.3.5 reports 0 rows.

I notice that the filename in the "Import Data" dialog seems to be missing the 
file
extension after selecting the file.

Original comment by djb...@gmail.com on 11 Aug 2008 at 6:51

[GoogleCodeExporter]

I tried several csv files and the import worked as expected. Could there be 
something
special about the file you are importing? If the data is not 
private/confidential,
can I get the file for testing?

As for display, extension is not shown. This is merely a display issue. In fact,
extension should be shown; or maybe, the whole path should be shown. Will fix 
it in
next release.

Original comment by mrinal.k...@gmail.com on 12 Aug 2008 at 3:35

[GoogleCodeExporter]

Also, I find that the following record gets imported correctly with version 
0.3.5:

VAUXHALL,"Vectra, MY2006","1.8i 16v 5 Door Hatchback with 15""/16""/17""
tyre, wide ratio gear box and Z18XE
engine",M5,1796,Petrol,10.4,5.5,7.3,27.1,51.3,38.6,175,1272,IV,72.0,0.232,0.020,
0.006

Original comment by mrinal.k...@gmail.com on 12 Aug 2008 at 3:40

[GoogleCodeExporter]

Attached file exhibits similar effects. I believe it is something to do with the
comma (or lack of it) at the end of the line.

Original comment by djb...@gmail.com on 12 Aug 2008 at 10:04

Attachments:

• glossary.csv

[GoogleCodeExporter]

1. Thanks for the file.
There is indeed a regression (in an attempt to handle records spanning multiple
lines) which I have corrected.
However, it appears that the lines 209 to 224 of the above file have a problem:
TD,turbo diesel,,
TDi,turbo charged direct injection diesel,,
TDI,turbo diesel with intercooler,,
4WD,4-wheel drive,,
4x4,4-wheel driveA,,automatic
A3,automatic 3-speed,,
A4,automatic 4-speed,,
A4 x 2,"ditto, high and low range gearing",,
A4L,"ditto, low ratio",,
ASM,automatic shift manual,,
AV,automatic variable transmission,,
AWD,all wheel drive,,
bhp,brake horsepower,,
CAT,catalyst,,
CVT,continuously variable transmission,,
D,diesel engined vehicle,,

From the number of commas it appears that there are 4 fields in the above lines.
However, all the other lines contain only two fields.

If these extra commas on these lines are an error in the csv file then I will 
push
the new version immediately. Otherwise, I must correct other errors too. 
Waiting for
your feedback.

Original comment by mrinal.k...@gmail.com on 13 Aug 2008 at 8:02

• Changed state: Accepted
• Added labels: Milestone-Release_0.3.6

[GoogleCodeExporter]

Original comment by mrinal.k...@gmail.com on 14 Aug 2008 at 10:00

• Changed state: Fixed

[GoogleCodeExporter]

Sorry for not replying to your earlier comment. I did not receive the email 
reminder 
from Google until you've fixed it.

Referring to the additional commas in comment 6 this is an unfortunate artefact 
of 
using Excel. When saving a spreadsheet in a CSV format it sometimes inserts 
additional commas if a column was used but subsequently cleared.

You have a number of options to handle this situation. One would be to stop 
processing at when a line has a different number of commas from the first line 
of 
the file and alert the user. This is how the SQLite shell programs works. 
Another 
(and this would be my prefer option) is to continue processing as long as there 
is 
no data in the additional commas and to inform the user that additional commas 
existed in the file (maybe identifying the first line of occurrence).

I'd be interested to read your feedback.

Original comment by djb...@gmail.com on 15 Aug 2008 at 9:22