search

conda-forge / osmnx-feedstock

A conda-smithy repository for osmnx.
BSD 3-Clause "New" or "Revised" License
8 stars 7 forks source link

The custom_filter results do not match the osm data. #71

Closed kolumdium closed 4 years ago

kolumdium commented 4 years ago

Issue: The custom_filter results do not match the actual osm data. More details below.


Environment (conda list):

``` (bicyclelogger) C:\Users\plank>conda list # packages in environment at C:\Users\plank\anaconda3\envs\bicyclelogger: # # Name Version Build Channel affine 2.3.0 py_0 attrs 20.2.0 py_0 backcall 0.2.0 py_0 blas 1.0 mkl bokeh 2.2.1 py38_0 branca 0.4.1 py_0 conda-forge brotlipy 0.7.0 py38he774522_1000 bzip2 1.0.8 he774522_0 ca-certificates 2020.6.20 hecda079_0 conda-forge certifi 2020.6.20 py38h9bdc248_2 conda-forge cffi 1.14.3 py38h7a1dbc1_0 cfitsio 3.470 he774522_6 chardet 3.0.4 py38_1003 click 7.1.2 py_0 click-plugins 1.1.1 py_0 cligj 0.5.0 py38_0 cloudpickle 1.6.0 py_0 colorama 0.4.3 py_0 cryptography 3.1.1 py38h7a1dbc1_0 curl 7.71.1 h4b64cdc_4 conda-forge cycler 0.10.0 py38_0 decorator 4.4.2 py_0 descartes 1.1.0 py_4 dill 0.3.2 py_0 expat 2.2.10 h33f27b4_2 fiona 1.8.13.post1 py38hd760492_0 flask 1.1.2 py_0 folium 0.11.0 py_0 conda-forge freetype 2.10.3 hd328e21_0 freexl 1.0.5 hfa6e2cd_0 gdal 3.0.4 py38h3ba59e7_9 conda-forge geographiclib 1.50 py_0 conda-forge geopandas 0.8.1 py_0 geopy 2.0.0 pyh9f0ad1d_0 conda-forge geos 3.8.1 h33f27b4_0 geotiff 1.5.1 h3d29ae3_10 conda-forge gettext 0.19.8.1 hb01d8f6_1002 conda-forge glib 2.65.0 he4de6d7_0 conda-forge hdf4 4.2.13 h712560f_2 hdf5 1.10.6 nompi_he0bbb20_101 conda-forge icc_rt 2019.0.0 h0cc432a_1 icu 64.2 he025d50_1 conda-forge idna 2.10 py_0 intel-openmp 2020.2 254 ipykernel 5.3.4 py38h5ca1d4c_0 ipython 7.18.1 py38h5ca1d4c_0 ipython_genutils 0.2.0 py38_0 itsdangerous 1.1.0 py_0 jedi 0.15.2 py38_0 conda-forge jinja2 2.11.2 py_0 joblib 0.17.0 py_0 jpeg 9d he774522_0 conda-forge jupyter_client 6.1.7 py_0 jupyter_core 4.6.3 py38_0 kealib 1.4.13 h3b59ab9_1 conda-forge kiwisolver 1.2.0 py38h74a9793_0 krb5 1.17.1 hc04afaa_0 libboost 1.67.0 hd9e427e_4 libcurl 7.71.1 h4b64cdc_4 conda-forge libffi 3.2.1 ha925a31_1007 libgdal 3.0.4 h6f60a84_9 conda-forge libiconv 1.15 h1df5818_7 libkml 1.3.0 he5f2a48_4 libnetcdf 4.7.4 nompi_h256d12c_105 conda-forge libpng 1.6.37 h2a8f88b_0 libpq 12.2 h3235a2c_0 libsodium 1.0.18 h62dcd97_0 libspatialindex 1.9.3 h33f27b4_0 libspatialite 4.3.0a h51df0ed_1038 conda-forge libssh2 1.9.0 h7a1dbc1_1 libtiff 4.1.0 h56a325e_1 libwebp-base 1.1.0 he774522_3 libxml2 2.9.10 h464c3ec_1 lz4-c 1.9.2 hf4a77e7_3 m2w64-expat 2.1.1 2 m2w64-gcc-libgfortran 5.3.0 6 m2w64-gcc-libs 5.3.0 7 m2w64-gcc-libs-core 5.3.0 7 m2w64-gettext 0.19.7 2 m2w64-gmp 6.1.0 2 m2w64-libiconv 1.14 6 m2w64-libwinpthread-git 5.0.0.4634.697f757 2 m2w64-xz 5.2.2 2 markupsafe 1.1.1 py38he774522_0 matplotlib-base 3.3.1 py38hba9282a_0 mkl 2020.2 256 mkl-service 2.3.0 py38hb782905_0 mkl_fft 1.2.0 py38h45dec08_0 mkl_random 1.1.1 py38h47e9c7a_0 mplleaflet 0.0.5 py_4 conda-forge msys2-conda-epoch 20160418 1 munch 2.5.0 py_0 networkx 2.5 py_0 numpy 1.19.1 py38h5510c5b_0 numpy-base 1.19.1 py38ha3acd2a_0 olefile 0.46 py_0 openjpeg 2.3.1 h57dd2e7_3 conda-forge openssl 1.1.1h he774522_0 conda-forge osmapi 1.2.2 py_0 conda-forge osmnx 0.16.1 pyh9f0ad1d_0 conda-forge packaging 20.4 py_0 pandas 1.1.3 py38ha925a31_0 parso 0.8.0 py_0 pcre 8.44 ha925a31_0 pickleshare 0.7.5 py38_1000 pillow 7.2.0 py38hcc1f983_0 pip 20.2.3 py38_0 poppler 0.87.0 h0cd1227_1 conda-forge poppler-data 0.4.9 1 conda-forge postgresql 12.2 h3235a2c_0 proj 7.0.0 haa36216_5 conda-forge prompt-toolkit 3.0.7 py_0 pycparser 2.20 py_2 pygments 2.7.1 py_0 pyopenssl 19.1.0 py_1 pyparsing 2.4.7 py_0 pyproj 2.6.1.post1 py38h1dd9442_0 conda-forge pysocks 1.7.1 py38_0 python 3.8.5 h5fd99cc_1 python-dateutil 2.8.1 py_0 python_abi 3.8 1_cp38 conda-forge pytz 2020.1 py_0 pywin32 227 py38he774522_1 pyyaml 5.3.1 py38he774522_1 pyzmq 19.0.2 py38ha925a31_1 rasterio 1.1.5 py38h151dc71_1 conda-forge requests 2.24.0 py_0 rtree 0.9.4 py38h21ff451_1 scikit-learn 0.23.2 py38h47e9c7a_0 scipy 1.5.0 py38h9439919_0 setuptools 50.3.0 py38h9490d1a_1 shapely 1.7.0 py38hbf43935_3 conda-forge six 1.15.0 py_0 snuggs 1.4.7 py_0 spyder-kernels 1.9.4 py38_0 sqlalchemy 1.3.19 py38he774522_0 sqlite 3.33.0 h2a8f88b_0 tabulate 0.8.7 py38_0 tbb 2020.3 h74a9793_0 threadpoolctl 2.1.0 pyh5ca1d4c_0 tiledb 1.7.7 h0b90766_3 conda-forge tk 8.6.10 he774522_0 tornado 6.0.4 py38he774522_1 traitlets 5.0.4 py_0 typing_extensions 3.7.4.3 py_0 urllib3 1.25.10 py_0 vc 14.1 h0510ff6_4 vincent 0.4.4 py_1 conda-forge vs2015_runtime 14.16.27012 hf0eaf9b_3 wcwidth 0.2.5 py_0 werkzeug 1.0.1 py_0 wheel 0.35.1 py_0 win_inet_pton 1.1.0 py38_0 wincertstore 0.2 py38_0 xerces-c 3.2.2 ha925a31_0 xz 5.2.5 h62dcd97_0 yaml 0.2.5 he774522_0 zeromq 4.3.2 ha925a31_3 zlib 1.2.11 h62dcd97_4 zstd 1.4.5 h04227a9_0 ```


Details about conda and system ( conda info ):

``` (bicyclelogger) C:\Users\plank>conda info active environment : bicyclelogger active env location : C:\Users\plank\anaconda3\envs\bicyclelogger shell level : 2 user config file : C:\Users\plank\.condarc populated config files : conda version : 4.8.5 conda-build version : 3.20.3 python version : 3.8.5.final.0 virtual packages : __cuda=11.1 base environment : C:\Users\plank\anaconda3 (writable) channel URLs : https://repo.anaconda.com/pkgs/main/win-64 https://repo.anaconda.com/pkgs/main/noarch https://repo.anaconda.com/pkgs/r/win-64 https://repo.anaconda.com/pkgs/r/noarch https://repo.anaconda.com/pkgs/msys2/win-64 https://repo.anaconda.com/pkgs/msys2/noarch package cache : C:\Users\plank\anaconda3\pkgs C:\Users\plank\.conda\pkgs C:\Users\plank\AppData\Local\conda\conda\pkgs envs directories : C:\Users\plank\anaconda3\envs C:\Users\plank\.conda\envs C:\Users\plank\AppData\Local\conda\conda\envs platform : win-64 user-agent : conda/4.8.5 requests/2.24.0 CPython/3.8.5 Windows/10 Windows/10.0.20231 administrator : False netrc file : None offline mode : False ```

Problem description

I was using the custom_filter option of the graph_from_place function when I noticed the retrieved data does not match the actual tags in osm. I used an area where I knew all tags had been set to either "residential" or "living_street". Here is how the overpass query should look like:

overpass

However here is what happens, If you query either tag individual you get a response. This response is already wrong though. Residential:

1

living_street:

2

If you use both with "|" you do not get streets with both tags:

4

You could use composed with networkx:

3

I am lost on why this is happening. I tracked down the query String and when you paste the result into Overpass you get the proper result So the query string seems fine:

query_str = f"{overpass_settings};(way{osm_filter}(poly:'{polygon_coord_str}');>;);out;"

[out:json][timeout:180];(way["highway"~"living_street|residential"](poly:'48.198563 11.498750 48.198745 11.498754 48.198909 11.498766 48.199071 11.498782 48.199192 11.498797 48.199203 11.498798 48.199320 11.498813 48.199378 11.498821 48.200344 11.498965 48.200360 11.498968 48.200649 11.499012 48.200905 11.499036 48.201023 11.499049 48.201417 11.499100 48.202188 11.499197 48.202593 11.499247 48.203278 11.499325 48.203332 11.499331 48.203750 11.499386 48.204192 11.499478 48.204626 11.499635 48.205047 11.499855 48.205452 11.500136 48.205836 11.500477 48.206196 11.500872 48.206527 11.501319 48.206827 11.501813 48.207094 11.502349 48.207323 11.502922 48.207513 11.503526 48.207662 11.504156 48.207769 11.504804 48.207832 11.505465 48.207851 11.506132 48.207849 11.506752 48.207825 11.507401 48.207759 11.508043 48.207652 11.508673 48.207506 11.509284 48.207320 11.509871 48.207196 11.510182 48.207150 11.510700 48.207101 11.511109 48.207066 11.511372 48.206999 11.511835 48.206968 11.512028 48.206499 11.514838 48.206480 11.514952 48.205841 11.518510 48.205837 11.518531 48.205595 11.519862 48.205604 11.519998 48.205608 11.520078 48.206888 11.520840 48.207967 11.521471 48.208074 11.521527 48.208278 11.521606 48.208655 11.521728 48.208945 11.521837 48.209194 11.521943 48.209247 11.521965 48.209433 11.521999 48.209509 11.522016 48.209895 11.522128 48.209921 11.522137 48.210345 11.522322 48.210754 11.522569 48.211146 11.522874 48.211516 11.523236 48.211860 11.523650 48.212175 11.524112 48.212459 11.524619 48.212708 11.525165 48.212920 11.525744 48.213093 11.526352 48.213225 11.526983 48.213315 11.527630 48.213362 11.528287 48.213367 11.528947 48.213343 11.529829 48.213301 11.530520 48.213209 11.531475 48.213136 11.532052 48.212995 11.532951 48.212893 11.533598 48.212892 11.533638 48.212832 11.534274 48.212732 11.534899 48.212593 11.535506 48.212416 11.536091 48.212253 11.536516 48.212444 11.536915 48.212667 11.537493 48.212851 11.538102 48.212994 11.538734 48.213094 11.539385 48.213150 11.540047 48.213162 11.540714 48.213130 11.541380 48.213054 11.542038 48.212935 11.542682 48.212773 11.543304 48.212572 11.543899 48.212331 11.544462 48.212173 11.544795 48.211912 11.545293 48.211621 11.545752 48.211302 11.546168 48.210959 11.546537 48.210594 11.546857 48.210210 11.547124 48.209811 11.547335 48.209401 11.547490 48.208983 11.547586 48.208561 11.547623 48.208138 11.547601 48.207437 11.547514 48.206813 11.547447 48.206153 11.547411 48.206101 11.547408 48.205638 11.547375 48.205627 11.547374 48.205480 11.547363 48.205443 11.547361 48.205398 11.547358 48.205295 11.547351 48.203858 11.547250 48.203661 11.547237 48.203605 11.547238 48.203517 11.547250 48.203477 11.547265 48.203049 11.547391 48.202615 11.547454 48.202179 11.547454 48.201745 11.547391 48.201317 11.547265 48.200899 11.547078 48.200496 11.546831 48.200446 11.546791 48.199677 11.546735 48.199671 11.546734 48.199007 11.546684 48.198867 11.546670 48.198634 11.546641 48.198452 11.546628 48.198368 11.546621 48.197088 11.546535 48.197081 11.546535 48.195303 11.546411 48.194307 11.546346 48.193911 11.546320 48.193658 11.546303 48.193451 11.546282 48.193125 11.546267 48.193050 11.546263 48.192584 11.546231 48.192515 11.546227 48.192327 11.546236 48.192235 11.546235 48.191789 11.546201 48.191348 11.546100 48.191322 11.546092 48.191279 11.546088 48.190469 11.546021 48.190456 11.546020 48.190258 11.546003 48.190069 11.545997 48.190059 11.545996 48.189778 11.545986 48.189744 11.546000 48.189353 11.546106 48.188958 11.546160 48.188896 11.546164 48.188451 11.546163 48.188008 11.546096 48.187573 11.545963 48.187147 11.545767 48.186737 11.545510 48.186346 11.545192 48.185978 11.544819 48.185637 11.544393 48.185325 11.543919 48.185046 11.543401 48.184803 11.542844 48.184597 11.542254 48.184432 11.541637 48.184308 11.540998 48.184229 11.540499 48.184187 11.540208 48.184123 11.539706 48.184113 11.539626 48.184060 11.539175 48.184041 11.539010 48.183986 11.538472 48.183956 11.538106 48.183914 11.537460 48.183896 11.537050 48.183880 11.536402 48.183881 11.535917 48.183896 11.535304 48.183915 11.534883 48.183957 11.534256 48.183990 11.533865 48.184066 11.533155 48.184158 11.532503 48.184292 11.531868 48.184329 11.531739 48.184348 11.531632 48.184553 11.530657 48.184676 11.530145 48.185003 11.528938 48.185296 11.527403 48.185421 11.526316 48.185523 11.525638 48.185672 11.524979 48.185866 11.524346 48.186017 11.523913 48.186155 11.523429 48.186263 11.523076 48.186823 11.521389 48.186869 11.521208 48.187067 11.520194 48.187214 11.519561 48.187351 11.519057 48.187723 11.517177 48.187868 11.516556 48.188141 11.515550 48.188316 11.514630 48.188421 11.514146 48.188792 11.512639 48.188851 11.512323 48.188921 11.511985 48.189450 11.509641 48.189667 11.508850 48.189875 11.508211 48.190064 11.507689 48.190283 11.507195 48.191113 11.505488 48.191379 11.504993 48.191956 11.504016 48.192033 11.503788 48.192121 11.503405 48.192279 11.502806 48.192475 11.502233 48.192706 11.501690 48.192972 11.501183 48.193268 11.500716 48.193593 11.500294 48.193944 11.499920 48.194316 11.499597 48.194708 11.499330 48.195115 11.499120 48.195534 11.498969 48.195960 11.498879 48.196663 11.498781 48.196716 11.498774 48.197391 11.498692 48.197847 11.498671 48.198302 11.498720 48.198395 11.498744 48.198554 11.498749 48.198563 11.498750');>;);out;

I tried to look where the error is comming from but I can not find it.

I would appriciate feedback. Am I just using it wrong or is this actually an issue? Also building on this are you sure your osm_filters from "_get_osm_filter" work correctly?

Thank you for your time. -Kolumdium

import osmnx as ox
import networkx as nx

placename = "Bezirksteil Lerchenau West, München"

tmp_graph = ox.graph_from_place(
    placename, network_type=None, 
    simplify = False, 
    custom_filter='["highway"~"residential"]')

fmap = ox.plot_graph_folium(tmp_graph, edge_color = "green", edge_width = 1)
fmap.save("{}.html".format(1))

tmp_graph2 = ox.graph_from_place(
    placename, network_type=None, 
    simplify = False, 
    custom_filter='["highway"~"living_street"]')

fmap = ox.plot_graph_folium(tmp_graph2, edge_color = "green", edge_width = 1)
fmap.save("{}.html".format(2))

tmp_graph3 = nx.compose(tmp_graph, tmp_graph2)

fmap = ox.plot_graph_folium(tmp_graph3, edge_color = "green", edge_width = 1)
fmap.save("{}.html".format(3))

tmp_graph4 = ox.graph_from_place(
    placename, network_type=None, 
    simplify = False, 
    custom_filter='["highway"~"living_street|residential"]')

fmap = ox.plot_graph_folium(tmp_graph4, edge_color = "green", edge_width = 1)
fmap.save("{}.html".format(4))
kolumdium commented 4 years ago

Sorry posted it to the wrong git.