search

timwis / vizwit

An interactive data visualization tool
http://vizwit.io
GNU General Public License v2.0
100 stars 35 forks source link

Filter out "magic" fields in socrata-fields collection #105

Open timwis opened 9 years ago

marks commented 8 years ago

My personal opinion is that empty data is data (the user should know the dataset has columns that have missing data). I think #123 takes care of this.. or maybe there is a config flag to opt into this. Just my 2c

timwis commented 8 years ago

I agree @marks -- this is actually a socrata-provider-specific issue: the fields/metadata endpoint gives back a bunch of those "magic" fields (the computed polygon ones, and like 5 location ones). There's not a clear way to filter them out when fetching the fields, so the table view renders them as columns. But when you get the actual data/rows, those fields don't exist, so they show up as empty columns. So I'll reword this issue to reference those magic fields (let me know if there's a more technical name for them)

marks commented 8 years ago

I believe we call these computed fields. Right now they only ever seem to have the format of :@computed_region_a1b2_c3d4 but I think it is safe to assume they will always start with :@ where : denotes a special column and @ denotes it is computed.

@timwis Shall we filter these columns out somewhere in the provider.js?

timwis commented 8 years ago

@marks good idea - it would be in socrata-fields.js, which is currently being refactored in #149.

The only other thing to look at is that socrata adds like 5 "location" fields too, and those seem to be blank as well. (location zip, location state, etc.)