More generic solution for fields collection

[timwis]

In an effort to abstract the data provider as much as possible, I've been thinking about whether the socrata-fields collection is necessary, and how it might be abstracted so there's just a socrata collection to establish the provider. Currently, the fields collection exists to get field aliases/friendly names (ie. Parcel Number instead of parcel_number) for use in (1) table column headers and (2) the "current filters" area below each card.

It was written as a distinct collection because the socrata provider requires a distinct request (to a separate endpoint) to get field names, and I felt it would be "cleaner" and more consistent than just using $.ajax() inside the socrata collection.

But I imagine that most providers don't provide a way to get field aliases/friendly names and, as such, we could probably just use the keys of the first row of data for them (unless the first record was missing some keys).

Ultimately, though, it feels like this should just be abstracted in the collection, like a getFields() method that returns a promise. That way anything calling getFields() doesn't have to know whether that promise called an ajax request behind the scenes or returned a static array.

@chriswhong @mheadd do CartoDB or Esri have distinct field-listing endpoints and/or aliases/friendly field names?

Note: at the moment, the table view also uses the fields model to set the default order of the table, but that's very socrata specific and will be removed in favor of setting it in the config file.

[mheadd]

There is no endpoint specifically to list fields that I am aware of in CKAN.

When using the SQL Search API to return results from a CKAN resource, the field listing is returned as part of the response. For example, here is a simple SQL query to return results for a dataset from Adams County, CO.

Worth noting that CKAN provides a bit more flexibility in naming fields - in the response for Adams County, note that some fields are already using a friendly name: Property Foreclosed and Lien Sent to Treasurer. CKAN will return the fields as they are already set up for that specific resource.

I will say that, though perhaps not ideal, if you did want to replicate what you are doing for Socrata - making a separate call to get field names - you could replicate that for a CKAN instance using the SQL search API by just using LIMIT = 0 which would just return a collection of field names.

That does feel kind of hacky though.

[timwis]

Thanks @mheadd - that's actually exactly what @chriswhong appears to have done in cartodb-fields.js

[mheadd]

Boom. There it is then.

I haven't started on a CKAN provider yet, but I hope to soon and can emulate this approach.

[timwis]

Yeah, thinking about it more, I think the implementation of "getting a list of fields" should be up to the provider -- if the socrata provider uses a separate collection file, it would require() it and the rest of the application would have no idea there was a separate collection file. Another provider might just use an $.ajax() call or a static array. The problem with this is that if you have 6 cards of the same dataset, you don't want to make 6 requests to get the fields (every card uses field names to print the current filters beneath it). This is why vizwit "shares" field collections in a "global" hash.

If we can think of an alternative way to do this architecturally, "providers" would feel a lot more distinct/abstracted. The only options coming to mind right now are (a) rely on the browser's built-in cache that prevents it making the same request twice, or (b) somehow put that "global" hash in the provider rather than the entire application.

Hopefully I'm articulating this well

[chriswhong]

+1 for building this into the providers so they are totally abstracted. I'll be hacking on this again this afternoon

On Friday, January 15, 2016, Tim Wisniewski notifications@github.com wrote:

Yeah, thinking about it more, I think the implementation of "getting a list of fields" should be up to the provider -- if the socrata provider uses a separate collection file, it would require() it and the rest of the application would have no idea there was a separate collection file. Another provider might just use an $.ajax() call or a static array. The problem with this is that if you have 6 cards of the same dataset, you don't want to make 6 requests to get the fields (every card uses field names to print the current filters beneath it). This is why vizwit "shares" field collections in a "global" hash.

If we can think of an alternative way to do this architecturally, "providers" would feel a lot more distinct/abstracted. The only options coming to mind right now are (a) rely on the browser's built-in cache that prevents it making the same request twice, or (b) somehow put that "global" hash in the provider rather than the entire application.

Hopefully I'm articulating this well

— Reply to this email directly or view it on GitHub https://github.com/timwis/vizwit/issues/147#issuecomment-171975319.