CSV previews

[timwis]

Options

A. Use recline.js (like CKAN) B. Roll something new using PapaParse and a grid library that handles large datasets C. Build a website like geojson.io for CSVs

Considerations

• Should load files even if CORS not available or different protocol (http on https). Perhaps a proxy like YQL?

[timwis]

See first comment in #15

[eeeschwartz]

@timwis perhaps the frontmatter could include an iframe preview url with sensible defaults to geojson.io (#15) or a recline service. But it could also allow a maintainer to point at the previewer of their choice.

To test out possibilities, I created a rough recline example that takes a dataset param like geojson does.

The idea that a maintainer could plug in their own previewer service could help JKAN to be extended nicely.

(And big high fives or this project. It's fantastic.)

[timwis]

Thanks @eeeschwartz! And that's an interesting idea. It may not be necessary to do it on a per-dataset level; rather, we could make the "csv previewer" and "geojson previewer" configurable at the site level. The key, I imagine, is to make it easier to plug in your own.

[eeeschwartz]

@timwis it's a good question. Not sure if people would always want different viewers or just in certain cases where the dataset is too big (maps) or hosted differently (google docs versus CKAN or whatever).

I was picturing that people would use the standard viewer 99% of the time until that especially squirrelly dataset comes along. Next step they create and inject the perfect viewer and they're giving toasts at the international IT awards :trophy:

But it's mostly speculation on my end. I'll get closer to my various use cases and see where that leads. Cheers

[keeganmcbride]

Hey @timwis have you had any other thoughts on how to approach this? We will start implementing the CSV previews this week on our end.

[timwis]

@keeganmcbride I have had some preliminary thoughts about this but not quite settled on a concrete idea. I was thinking it may be worth it to build a service similar to geojson.io but for CSV files which provides an embeddable table. The primary reason for the service would be to add CORS headers. Alternatively, if the JKAN site hosts the CSV files on the same origin or on one that already has CORS headers, we could fetch the data directly. I reckon something like papa parse's streaming method would be good for pulling in large files without crashing the browser tab. What were you thinking about?

[keeganmcbride]

@timwis We decided to just go with a super simple solution (could definitely be improved) that basically just uses OKFN's datapipes for the preview functionality. See, for example, here: https://opendata.riik.ee/andmehulgad/avaliku-korra-vastased-ja-avalikus-kohas-toime-pandud-syyteod/ shout out to the San Diego OGD portal for inspiration for some of the UI