search

hasadna / datacity-k8s

1 stars 0 forks source link

export all data #125

Open niryariv opened 3 years ago

niryariv commented 3 years ago

Client asked for the ability to store a local copy of all the data on their instance.

  1. Is there a tool that simply exports all CKAN datasets as a zip file or something similar?

  2. If not, can you estimate time investment to build one? It should be something accessible to admin, eg "Download all data" button that packages everything and downloads it to their desktop.

OriHoch commented 3 years ago
  1. couldn't find an existing tool
  2. not hard to do it, need to decide more specifically what data will be contained in this zip, what about metadata, where / how to run it - I think it's best if we could run background processing tasks on demand (related to #129)
akariv commented 3 years ago

I think we could have a weekly task to zip everything on an instance and store the file on a bucket, and in the instance itself ust have a link to a static asset. It's simpler and fulfills the requirement here imo.

On Wed, Nov 11, 2020 at 2:58 PM Ori Hoch notifications@github.com wrote:

  1. couldn't find an existing tool
  2. not hard to do it, need to decide more specifically what data will be contained in this zip, what about metadata, where / how to run it - I think it's best if we could run background processing tasks on demand (related to #129 https://github.com/hasadna/datacity-k8s/issues/129)

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/hasadna/datacity-k8s/issues/125#issuecomment-725409150, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACAY5IC2ZX33KQMGINRLZDSPKC77ANCNFSM4TPHK75Q .

niryariv commented 3 years ago

Moving this to backlog for now, as it seems it might no longer be a requirement from Kfar Saba at the moment.