whosonfirst-data / whosonfirst-data-postalcode-at

Who's On First postal code data for AT
https://whosonfirst.mapzen.com
Other
1 stars 0 forks source link

Austrian Zips (Postleitzahlen) #1

Closed kevinkreiser closed 8 years ago

kevinkreiser commented 8 years ago

Firstly, all of the data is licensed as: http://creativecommons.org/licenses/by/3.0/at/

So the Austrian government gives out the polygons for their zip codes. They come as shapefiles here:

https://www.data.gv.at/katalog/dataset/c33d36b0-f184-4f2a-89cc-839ca7fcf88a

Each polygon has a Gemeindekennziffer or GKZ (ID), names and some other attributes in a shapefile but not the actual zip code..

But the statistics branch of their govt produces a csv file which has a column for the GKZ and the actual zip code. The csv is here:

http://www.statistik.at/web_de/klassifikationen/regionale_gliederungen/gemeinden/index.html

So now we must join them together. I wrote a script to do this, its pretty elementary so don't LOL too hard:

#!/bin/bash
set -e

#get the zip code shapes into pretty printed geojson
if [ ! -e gem-gr-at-15-18.zip ]; then
  wget http://ckan.data.ktn.gv.at/storage/f/2015-06-30T08%3A06%3A45.158Z/gem-gr-at-15-18.zip
fi
unzip -qq gem-gr-at-15-18.zip
ogr2ogr -f GeoJSON -t_srs crs:84 at_gkz_singleline.geojson Gem_AT_2015.shp
cat at_gkz_singleline.geojson | python -m json.tool > at_gkz.geojson

#get the csv with the actual zip codes in it
if [ ! -e gemliste_knz.csv ]; then
  wget http://www.statistik.at/verzeichnis/reglisten/gemliste_knz.csv
fi
iconv -f ISO-8859-14 -t UTF-8 gemliste_knz.csv > gemliste_knz_utf8.csv

#join the shape with the zip codes via the Gemeindekennziffer found in both
cat at_gkz.geojson | while read line; do
  if [[ "${line}" == *"GEM_NR"* ]]; then
    gkz=$(echo "${line}" | sed -e "s/.*GEM_NR\": \([0-9]\{5\}\).*/\1/g")
    #wien has 23 zips we'll have to get their shapes from another file
    #for now we cram them into one polygon that encompases them all
    #yes this means that each zip is an array of numbers (only wien has more than 1)
    plz=$(grep -E "^${gkz};" gemliste_knz_utf8.csv | awk -F';' '{print $5","}' | tr -d '\n' | sed -e "s/^/[/g" -e "s/,$/]/g")
    echo "\"gkz\":${gkz},\"plz\":${plz},"
  else
    echo "${line}"
  fi
done > at_plz_ugly.geojson
cat at_plz_ugly.geojson | python -m json.tool > at_plz.geojson
GZIP=-9 gzip at_plz.geojson

I don't know if any of this works on mac or whatever, I've only ran this on ubuntu (sed is known to not be the same everywhere..). So I've attached the output of this to the issue PM me and I can just send you the file directly. The column with the zip code in it is named plz.

The main caveat to this.. The capitol, Vienna/Wien, is divided up into 23 further zip codes, each of which has a 1 as the first digit and a 0 as the last digit. A single polygon is included in this file that encompasses all 23 zips so we cram them all into that one. I'm pretty sure I can get these geometries separately (I think I saw them on the govt download site). I'll mess with that in another issue.

thisisaaronland commented 8 years ago

screen shot 2016-05-09 at 13 26 20 screen shot 2016-05-09 at 13 28 54

thisisaaronland commented 8 years ago

I am going to close this issue and open new ones for Wien and the two records with bunk geometries:

seleqman commented 6 years ago

Hello guys. I am searching google up and down for 3 days and still nothing usefull. Don't you have .shp of austria and poland zip codes polygons? thank you.

kevinkreiser commented 6 years ago

@seleqman I'm not too familiar with how WOF works at the moment but I think it gives you geojson geometries for everything. From there you could easily convert to shp if you want to. perhaps @thisisaaronland can point you to the relevant geojson?

thisisaaronland commented 6 years ago

@seleqman There are not published shapefiles, although that's on the list. In the interim the solution is to produce a GeoJSON FeatureCollection from the source files and then use ogr2ogr to convert it in to a shapefile.

You can either "roll your own" tool to generate the FeatureCollection or do the following:

cd whosonfirst-data-postalcode-au
make metafiles
...time passes and ./meta/wof-postalcode-au-latest.csv is created 
cd ../
git clone git@github.com:whosonfirst/py-mapzen-whosonfirst-utils.git
cd py-mapzen-whosonfirst-utils
make install <-- warning this has a maze of twisty dependencies...
cd ../
/usr/local/bin/wof-csv-to-feature-collection --source whosonfirst-data-postalcode-au --csv whosonfirst-data-postalcode-au/meta/wof-postalcode-au-latest.csv --out wof-postalcode-au-latest.geojson

Which, you know... isn't awesome. These are a bunch of tools that have been written over the years for internal work to account for specific tasks (trimmed down properties, etc.) that could do what you're after, but honestly a hand-written tool might be easier.

A brief search suggests there is a pure-Go shapefile writer which if it works would be super useful (less dependency hoohah than with the Python tools) so I'll try to look at that soon.

therealphinz commented 4 days ago

I collected all Datapoints (BKZ, GKZ, Districts, Municipalities, Federal states, Postal codes, Geometries of ZIPs ) in that CSV file. Dataset.csv for AUSTRIA image