Preview of data

[stevieflow]

Is there a config in DKAN to enable preview of CSV files, for example?

I note that the Trafford data does not have preview:

http://data.threesixtygiving.org/dataset/grants-voluntary-and-community-organisations/resource/7dfe26ac-4afd-4957-96b7-a2fa9651f61f

Over at DataGM (a CKAN), there is:

http://datagm.org.uk/dataset/transparency-code-trafford/resource/2927896b-0f07-475a-8ce4-5366b38a4ae2

[practicalparticipation]

There should be a Manage Datastore tab for users with the right permissions, with an option to import the data.

I believe this is manually run, so we may need to look at how this would happen as part of scheduled jobs.

On Wed, Mar 11, 2015 at 8:41 AM, Steven Flower notifications@github.com wrote:

Is there a config in DKAN to enable preview of CSV files, for example?

I note that the Trafford data does not have preview:

http://data.threesixtygiving.org/dataset/grants-voluntary-and-community-organisations/resource/7dfe26ac-4afd-4957-96b7-a2fa9651f61f

Over at DataGM (a CKAN), there is:

http://datagm.org.uk/dataset/transparency-code-trafford/resource/2927896b-0f07-475a-8ce4-5366b38a4ae2

— Reply to this email directly or view it on GitHub https://github.com/ThreeSixtyGiving/data.threesixtygiving.org/issues/13.

w: http://www.timdavies.org.uk | m: 07834 856 303 | twitter: timdavies

Co-director of Practical Participation:

http://www.practicalparticipation.co.uk

Practical Participation Ltd is a registered company in England and Wales -

5381958.

[stevieflow]

Great, thanks @practicalparticipation . Apologies - some of these features are new to me

The Macc data doesnt seem to import, perhaps because it is a .xlsx / multisheet file:

http://data.threesixtygiving.org/dataset/macc-grants/resource/c3a30bff-f1b4-453c-bea7-0a88b5a5e966

How best to do that?

[pmackay]

@stevieflow can the data be saved out to a basic CSV? Not sure DKAN would handle .xlsx...

[stevieflow]

@pmackay thanks. It has two worksheets as they have multiple admin geography codes (wards) for each grant. Saving an .xlsx as a csv will only flatten/save one of the sheets, wouldn't it?

[practicalparticipation]

So this is where DKAN will only be able to handle flat data, and we'll need a more advanced data store to ingest the JSON versions of files and provide better access to them.

That is something we should think about putting on the workplan as resources become available.

On Wed, Mar 11, 2015 at 1:47 PM, Steven Flower notifications@github.com wrote:

@pmackay https://github.com/pmackay thanks. It has two worksheets as they have multiple admin geography codes (wards) for each grant. Saving an .xlsx as a csv will only flatten/save one of the sheets, wouldn't it?

— Reply to this email directly or view it on GitHub https://github.com/ThreeSixtyGiving/data.threesixtygiving.org/issues/13#issuecomment-78265424 .

w: http://www.timdavies.org.uk | m: 07834 856 303 | twitter: timdavies

Co-director of Practical Participation:

http://www.practicalparticipation.co.uk

Practical Participation Ltd is a registered company in England and Wales -

5381958.

[pmackay]

Yes only way with DKAN would be to save to 2 CSV files and import those. For the purpose of data indexing, etc is that worth doing, or more effort than its worth?

[stevieflow]

Hmm - struggled to generate a preview on the two latest datasets:

http://data.threesixtygiving.org/dataset/department-social-development-northern-ireland-government-grants-voluntary-and-community-1

http://data.threesixtygiving.org/dataset/welcome-trust-grants-awarded-1-october-2013-%E2%80%93-30-september-2014/resource/7e93f601-dfb8-4526

I tried the Manage Datastore > Import option with both csv files

[practicalparticipation]

They appear to be available via the Datastore API:

http://data.threesixtygiving.org/api/action/datastore/search.json?resource_id=7e93f601-dfb8-4526-a672-2374de464b40&limit=5

http://data.threesixtygiving.org/api/action/datastore/search.json?resource_id=ab67d2b2-0f15-47d1-ac09-1c30b7b7efc3&limit=5

So the import worked, but for some reason they are not displaying in the previewer. Not sure why...

On Mon, Mar 16, 2015 at 11:25 AM, Steven Flower notifications@github.com wrote:

Hmm - struggled to generate a preview on the two latest datasets:

http://data.threesixtygiving.org/dataset/department-social-development-northern-ireland-government-grants-voluntary-and-community-1

http://data.threesixtygiving.org/dataset/welcome-trust-grants-awarded-1-october-2013-%E2%80%93-30-september-2014/resource/7e93f601-dfb8-4526

I tried the Manage Datastore > Import option with both csv files

— Reply to this email directly or view it on GitHub https://github.com/ThreeSixtyGiving/data.threesixtygiving.org/issues/13#issuecomment-81589741 .

w: http://www.timdavies.org.uk | m: 07834 856 303 | twitter: timdavies

Co-director of Practical Participation:

http://www.practicalparticipation.co.uk

Practical Participation Ltd is a registered company in England and Wales -

5381958.