search

hasadna / datacity-k8s

1 stars 0 forks source link

Make sure the system doesn't auto-create files when the format already exists #191

Open niryariv opened 3 years ago

niryariv commented 3 years ago
  1. User uploaded CSV + XLS files: https://kfar-saba.datacity.org.il/dataset/yoatzim
  2. The system auto-created an extra, redundant XLS file: Screen Shot 2021-02-17 at 15 24 23

As usual please give me a time estimate for now

OriHoch commented 3 years ago

how do you suggest to fix it? how will the automated process detect this case and decide when to skip generation of xls file?

niryariv commented 3 years ago

I assumed something like "if an xls file exists, do not create xls file".. What am I missing?

OriHoch commented 3 years ago

no problem, that's one way to do it, bear in mind there could be many different files in 1 resource

so if there are any .xls or .xslx file in the resource, don't create another one?

niryariv commented 3 years ago

yes, checking by the filename. Ie, if the following exist:

a.csv
b.csv
a.xls
b1.xls

create b.xls only

OriHoch commented 3 years ago

ok, 1-2 hours

niryariv commented 3 years ago

thanks. moving to to do for now