search

OCHA-DAP / Data-Team

A place for tracking data team issues
0 stars 1 forks source link

Some IPI resources are too large for preview #89

Open cjhendrix opened 9 years ago

cjhendrix commented 9 years ago

Some of the resources here are not previewing because the file size is quite large (47mb). https://data.hdx.rwlabs.org/dataset/ipi-peacekeeping-database/resource/64e06182-d4a9-4739-b0a5-02b14d8f3a9d

Options:

  1. Would it be possible to make these files smaller by cutting unneeded data?
  2. Zip them (which disables the preview button)
  3. Load them via the api to datastore (not sure this would fix it, checking with rufus)

Making dataproxy accept larger files sizes is probably not desirable because there would be a long wait while the file is uploaded to dataproxy and downloaded as json.

image

tagging @luiscape

luiscape commented 9 years ago

@cjhendrix , here is a quick follow-up:

  1. I am not sure there is unnecessary data there. I simply linked to the original datasets hosted on their website.
  2. I can ZIP them, but then it seems a bit of a cumbersome change for our users. The data is available in CSV on their site. So it seems to be extra work for our users to actually get to the data.
  3. I can certainly give it a try. However, i'm not sure it's working with the new deployment (v0.4.0)... will follow-up.