ThreeSixtyGiving / standard

The 360Giving data standard for UK philanthropic giving
http://www.threesixtygiving.org
Other
10 stars 15 forks source link

Dots in (publisher own) field titles can create problems #223

Closed BobHarper1 closed 6 years ago

BobHarper1 commented 6 years ago

We've seen cases where publishers using additional fields of their own use dots in the title e.g. field.name

While this shouldn't be expected to be a problem it has created issues when reprocessing data (in one case, mapping to an Elasticsearch index rejected those grant records in full). Also, dot notation is used in javascript to find properties of objects. So in this example an application might read field.name as name, a property of field, causing an error.

At the moment in the documentation we say:

If you have additional data to report that does not fit any of the columns provided in the spreadsheet, it is okay to create your own column titles in order to report it.

Proposal is to amend this bit so that it's clear that publishers should not use dots in those extra column titles.

robredpath commented 6 years ago

Proposal is to amend this bit so that it's clear that publishers should not use dots in those extra column titles.

Perhaps we could make recommendations as to good characters to use? Maybe say "It is recommended that additional column titles use only the characters used by the 360Giving standard (A-Z, a-z, 0-9, : , (), ?), spaces, dashes or underscores"? I think, and I'm sure @Bjwebb or @kindly will correct me if I'm wrong, but that would give a set that's likely to be well-supported by tools, unlikely to require changes by most publishers, and gives plenty of room for creation of helpful titles.

BobHarper1 commented 6 years ago

@robredpath That's a good approach. Column titles there may be potentially more problematic characters than the few we identified so far, so giving a reasonable but assured set should be helpful, rather than a hindrance.

Field values can be more permissive (except in the case of identifiers, but they do have a standard methodology already outlined in the documentation).

BobHarper1 commented 6 years ago

We've now included this guidance in the documentation (and the non-dependent issue in grantnav has also been addressed).