Mapnik CSV error handling

[who8mycakes]

Breakdown of the problem

An instance of a csv with an empty string erred because the error message was > 256 characters.

The headers in the file triggered the error because the index column was an empty string, which violates a check in Mapnik.

What could be fixed/addressed?

• A limit for size/length of an error message string.
• The error message delivered from Mapnik: shorten the error by removing - this is required for reading geometry data from the error message, and/or truncate all error messages to 256 characters.

Technical approach

• Going to take a moderate approach at this stage, and fix the error message bug within Mapnik-omnivore. Later on we can formulate an approach to strict CSV validation.

Dependencies:

The changes to Mapnik-omnivore are a bugfix and won't be a breaking change, and will merely be a patch release.

Questions:

• Do we want to take a duct-tape approach and just get this "working", or do we want to take the marathon route and also incorporate strict parsing mode for CSV files?
• In such instances as these, should this be considered a valid or invalid csv?

/cc @springmeyer @GretaCB @artemp @mapsam @flippmoke

[artemp]

@who8mycakes - in my view we should be able to process this file replacing an empty column name with automatically generated name e.g "_1" . I'm going to look at fixing this in mapnik core.

I'm also in favour of truncating redundant error message: "this is required for reading geometry data" is not adding anything and should be removed.

[artemp]

https://github.com/mapnik/mapnik/commit/7c6f335fa5b2194a2d52a6b78d2c02f056c29289 /cc @who8mycakes @springmeyer @GretaCB @mapsam @flippmoke

[mapsam]

CSV errors are being handled in mapnik-omnivore now! Updates in mapnik will be great whenever node-mapnik is released. Considering this closed now.