search

OpenDataServices / cove

CoVE is an web application to Convert, Validate and Explore data following certain open data standards - including 360Giving, Open Contracting Data Standard, IATI and the Beneficial Ownership Data Standard
http://cove.opendataservices.coop
Other
43 stars 11 forks source link

Spreadsheets with slashes in titles- better error message? #166

Closed caprenter closed 8 years ago

caprenter commented 8 years ago

To reproduce;

On 360Giving version. Use this link: http://data.threesixtygiving.org/sites/default/files/IndigoTrust_360giving.csv

You should get this error: Sorry we can't process that data

We think you tried to supply a spreadsheet, but we failed to convert it to JSON.

Error message: ValueError('Unrecognised type: "County/State"',)

Is there a way to make this error message more useful? In this case the column that fails is "Recipient Org:County/State"

I think it's ok for CoVE to fail here and inform the user - I don't think CoVE needs to be more forgiving in this case.

timgdavies commented 8 years ago

@Bjwebb Is this being caused by flatten-tool handling of '/' of elsewhere in CoVE validation?

If so, we might want to make flatten-tool more forgiving of illegal characters in field names, or otherwise report the error in a friendly way.

Bjwebb commented 8 years ago

This is how the flattentool handles /.

I think more friendly error report is probably the way forward here (making slashes work in titles would require some other way to distinguish titles vs. fields).

Bjwebb commented 8 years ago

The underlying error here should be fixed by these flatten-tool changes https://github.com/OpenDataServices/flatten-tool/pull/89

Bjwebb commented 8 years ago

Running this against a copy of Cove with those flatten-tool changes http://flatten-tool-updates.dev.cove.opendataservices.coop/360/?source_url=http://data.threesixtygiving.org/sites/default/files/IndigoTrust_360giving.csv

We no longer get the / error, but a different one (due to a different header) instead:

We think you tried to supply a spreadsheet, but we failed to convert it to JSON.

Error message: ValueError("There is an object or list at 'awardDate' but it should be an string",)