hasadna / datacity-k8s

1 stars 0 forks source link

זהוי אוטומטי של שדות קורדינטות #146

Closed niryariv closed 3 years ago

niryariv commented 3 years ago

if the file contains fields with the following names assume this is geodata and handle accordingly (geojson, ITM transform):

lat | lon lat | long latitude | longtitude x | y e_ord | n_ord

OriHoch commented 3 years ago

@niryariv the x and y seems to have been switched in the table, I think this is the correct ordering:

lat | lon lat | long latitude | longtitude y | x

how does e_ord | n_ord relate to this? which one is lat / lon?

niryariv commented 3 years ago

Latitude = Y = N_ORD Longitude = X = E_ORD

OriHoch commented 3 years ago

fixed in https://github.com/hasadna/datacity-ckan-dgp/commit/ba132dd36dfc07b10b216e3f74b3c15399110631

OriHoch commented 3 years ago

I ran this on Jerusalem, and encountered a problem where multiple geojson resources were added to a package, because it processes all relevant resources, and in many cases there are both csv and xlsx resources with the same data

This can occur both for the automatic xlsx resources we generate and for cases where user manually uploads multiple formats of a file.

We need to determine a way to prevent this.

I manually deleted all the duplicated geojson resources and disabled this feature for now.

OriHoch commented 3 years ago

@niryariv assigning to you, please prioritize

niryariv commented 3 years ago

Is there a way for us to determine / flag an auto generated file? Generally I instruct users that files in a dataset should contain the same data in various formats (TBH not sure if that's the standard practice) so we can assume a single geojson file is enough

OriHoch commented 3 years ago

I can identify auto-generated files, the problem is with manually uploaded files, there could be multiple resources with different data, or same data, we can't know for sure

niryariv commented 3 years ago

for now let's assume the data in all files is the same. so we need to (1) check if a geojson exists, and (2) if not, convert one file - preferably the CSV version

niryariv commented 3 years ago

for now let's assume the data in all files is the same. so we need to (1) check if a geojson exists, and (2) if not, convert one file - preferably the CSV version

can we implement this for now?

OriHoch commented 3 years ago

yes, so move this issue to high priority?

OriHoch commented 3 years ago

done, deployed

resource updated - https://jerusalem.datacity.org.il/dataset/recycling-facilities

niryariv commented 3 years ago

also add POINT_X (lon) | POINT_Y (lat)

niryariv commented 3 years ago

להוסיף את השדות הבאים לצורך הקובץ הזה (מעלה צילום כי גיטהאב לא מסתדר עם שילוב עברית-אנגלית):

Screen Shot 2021-05-05 at 13 11 07

niryariv commented 3 years ago

wontfix for now