OSMnx returning "Found no graph nodes within the requested polygon" incosistently when working with counties using Census shape files

[MaherSaid]

Making this report based on my post on StackOverflow (now deleted following the discussion in this thread) as I believe this may be a bug.

Problem description I'm preparing code to download street network for the Bay Area based on the following example. My version of the code (with small modifications) is at the end of this report. The code returns an error for both, the original example and my version, given the entire Bay Area. However, it works okay if we comment out some of the counties as such:

bayarea = {#'Alameda':'001',
           'Contra Costa':'013',
           'Marin':'041',
           'Napa':'055',
           'San Francisco':'075',
           #'San Mateo':'081',
           #'Santa Clara':'085',
           'Solano':'095',
           'Sonoma':'097'}

I'm not sure what the issue is, as it seems inconsistent. Uncommenting any of San Mateo, Santa Clara or Alameda in the dictionary above results in the code erroring out too for example. Any suggestions on how to go about resolving this? This is not exclusive to the Bay Area, but seems to happen for other cities when combining counties in the same fashion.

Here's the error (with the last few relevant lines from log):

2022-08-01 15:30:53 Got all network data within polygon from API in 21 request(s)
2022-08-01 15:30:53 Creating graph from downloaded OSM data...
2022-08-01 15:31:24 Created graph with 1501933 nodes and 2799609 edges
2022-08-01 15:31:39 Added length attributes to graph edges
2022-08-01 15:31:40 Identifying all nodes that lie outside the polygon...
2022-08-01 15:32:30 Created nodes GeoDataFrame from graph
2022-08-01 15:32:53 Created r-tree spatial index for 1501933 geometries
2022-08-01 15:33:01 Identified 0 geometries inside polygon
Traceback (most recent call last):
  File "/mnt/lib/test.py", line 44, in <module>
    G = ox.graph_from_polygon(bayarea_polygon, network_type='drive', simplify=True)
  File "/home/user/miniconda3/envs/osmnx/lib/python3.10/site-packages/osmnx/graph.py", line 430, in graph_from_polygon
    G_buff = truncate.truncate_graph_polygon(G_buff, poly_buff, True, truncate_by_edge)
  File "/home/user/miniconda3/envs/osmnx/lib/python3.10/site-packages/osmnx/truncate.py", line 161, in truncate_graph_polygon
    raise ValueError("Found no graph nodes within the requested polygon")
ValueError: Found no graph nodes within the requested polygon

Environment information

• OS: Ubuntu 18.04.5 LTS
• Python: 3.10.5
• OSMnx: 1.1.1 & 1.2.1
• Conda env:

``` _libgcc_mutex 0.1 conda_forge conda-forge _openmp_mutex 4.5 2_gnu conda-forge affine 2.3.1 pyhd8ed1ab_0 conda-forge attrs 22.1.0 pyh71513ae_1 conda-forge blosc 1.21.1 h83bc5f7_3 conda-forge boost-cpp 1.74.0 h75c5d50_8 conda-forge branca 0.5.0 pyhd8ed1ab_0 conda-forge brotli 1.0.9 h166bdaf_7 conda-forge brotli-bin 1.0.9 h166bdaf_7 conda-forge brotlipy 0.7.0 py310h5764c6d_1004 conda-forge bzip2 1.0.8 h7f98852_4 conda-forge c-ares 1.18.1 h7f98852_0 conda-forge ca-certificates 2022.6.15 ha878542_0 conda-forge cairo 1.16.0 ha61ee94_1011 conda-forge certifi 2022.6.15 py310hff52083_0 conda-forge cffi 1.15.1 py310h255011f_0 conda-forge cfitsio 4.1.0 hd9d235c_0 conda-forge charset-normalizer 2.1.0 pyhd8ed1ab_0 conda-forge click 8.1.3 py310hff52083_0 conda-forge click-plugins 1.1.1 py_0 conda-forge cligj 0.7.2 pyhd8ed1ab_1 conda-forge cryptography 37.0.4 py310h597c629_0 conda-forge curl 7.83.1 h7bff187_0 conda-forge cycler 0.11.0 pyhd8ed1ab_0 conda-forge expat 2.4.8 h27087fc_0 conda-forge fiona 1.8.21 py310h60a68a4_2 conda-forge folium 0.12.1.post1 pyhd8ed1ab_1 conda-forge font-ttf-dejavu-sans-mono 2.37 hab24e00_0 conda-forge font-ttf-inconsolata 3.000 h77eed37_0 conda-forge font-ttf-source-code-pro 2.038 h77eed37_0 conda-forge font-ttf-ubuntu 0.83 hab24e00_0 conda-forge fontconfig 2.14.0 h8e229c2_0 conda-forge fonts-conda-ecosystem 1 0 conda-forge fonts-conda-forge 1 0 conda-forge fonttools 4.34.4 py310h5764c6d_0 conda-forge freetype 2.10.4 h0708190_1 conda-forge freexl 1.0.6 h7f98852_0 conda-forge gdal 3.5.1 py310hb7951cf_2 conda-forge geopandas 0.11.1 pyhd8ed1ab_0 conda-forge geopandas-base 0.11.1 pyha770c72_0 conda-forge geos 3.11.0 h27087fc_0 conda-forge geotiff 1.7.1 h4fc65e6_3 conda-forge gettext 0.19.8.1 h73d1719_1008 conda-forge giflib 5.2.1 h36c2ea0_2 conda-forge hdf4 4.2.15 h10796ff_3 conda-forge hdf5 1.12.2 nompi_h2386368_100 conda-forge icu 70.1 h27087fc_0 conda-forge idna 3.3 pyhd8ed1ab_0 conda-forge jinja2 3.1.2 pyhd8ed1ab_1 conda-forge joblib 1.1.0 pyhd8ed1ab_0 conda-forge jpeg 9e h166bdaf_2 conda-forge json-c 0.16 hc379101_0 conda-forge kealib 1.4.15 ha7026e8_1 conda-forge keyutils 1.6.1 h166bdaf_0 conda-forge kiwisolver 1.4.4 py310hbf28c38_0 conda-forge krb5 1.19.3 h3790be6_0 conda-forge lcms2 2.12 hddcbb42_0 conda-forge ld_impl_linux-64 2.36.1 hea4e1c9_2 conda-forge lerc 4.0.0 h27087fc_0 conda-forge libblas 3.9.0 15_linux64_openblas conda-forge libbrotlicommon 1.0.9 h166bdaf_7 conda-forge libbrotlidec 1.0.9 h166bdaf_7 conda-forge libbrotlienc 1.0.9 h166bdaf_7 conda-forge libcblas 3.9.0 15_linux64_openblas conda-forge libcurl 7.83.1 h7bff187_0 conda-forge libdap4 3.20.6 hd7c4107_2 conda-forge libdeflate 1.12 h166bdaf_0 conda-forge libedit 3.1.20191231 he28a2e2_2 conda-forge libev 4.33 h516909a_1 conda-forge libffi 3.4.2 h7f98852_5 conda-forge libgcc-ng 12.1.0 h8d9b700_16 conda-forge libgdal 3.5.1 h53011df_2 conda-forge libgfortran-ng 12.1.0 h69a702a_16 conda-forge libgfortran5 12.1.0 hdcd56e2_16 conda-forge libglib 2.72.1 h2d90d5f_0 conda-forge libgomp 12.1.0 h8d9b700_16 conda-forge libiconv 1.16 h516909a_0 conda-forge libkml 1.3.0 h238a007_1014 conda-forge liblapack 3.9.0 15_linux64_openblas conda-forge libnetcdf 4.8.1 nompi_h21705cb_103 conda-forge libnghttp2 1.47.0 h727a467_0 conda-forge libnsl 2.0.0 h7f98852_0 conda-forge libopenblas 0.3.20 pthreads_h78a6416_1 conda-forge libpng 1.6.37 h753d276_3 conda-forge libpq 14.4 hd77ab85_0 conda-forge librttopo 1.1.0 hf730bdb_11 conda-forge libspatialindex 1.9.3 h9c3ff4c_4 conda-forge libspatialite 5.0.1 h38b5f51_18 conda-forge libssh2 1.10.0 ha56f1ee_2 conda-forge libstdcxx-ng 12.1.0 ha89aaad_16 conda-forge libtiff 4.4.0 h0d92c0b_2 conda-forge libuuid 2.32.1 h7f98852_1000 conda-forge libwebp 1.2.3 h522a892_1 conda-forge libwebp-base 1.2.3 h166bdaf_2 conda-forge libxcb 1.13 h7f98852_1004 conda-forge libxml2 2.9.14 h22db469_3 conda-forge libzip 1.9.2 hc869a4a_0 conda-forge libzlib 1.2.12 h166bdaf_2 conda-forge lz4-c 1.9.3 h9c3ff4c_1 conda-forge mapclassify 2.4.3 pyhd8ed1ab_0 conda-forge markupsafe 2.1.1 py310h5764c6d_1 conda-forge matplotlib-base 3.5.2 py310h5701ce4_1 conda-forge munch 2.5.0 py_0 conda-forge munkres 1.1.4 pyh9f0ad1d_0 conda-forge ncurses 6.3 h27087fc_1 conda-forge networkx 2.8.5 pyhd8ed1ab_0 conda-forge nspr 4.32 h9c3ff4c_1 conda-forge nss 3.78 h2350873_0 conda-forge numpy 1.23.1 py310h53a5b5f_0 conda-forge openjpeg 2.4.0 hb52868f_1 conda-forge openssl 1.1.1q h166bdaf_0 conda-forge osmnx 1.2.1 pyhd8ed1ab_0 conda-forge packaging 21.3 pyhd8ed1ab_0 conda-forge pandas 1.4.3 py310h769672d_0 conda-forge pcre 8.45 h9c3ff4c_0 conda-forge pillow 9.2.0 py310he619898_0 conda-forge pip 22.2.1 pyhd8ed1ab_0 conda-forge pixman 0.40.0 h36c2ea0_0 conda-forge poppler 22.04.0 h1434ded_1 conda-forge poppler-data 0.4.11 hd8ed1ab_0 conda-forge postgresql 14.4 hfdbbde3_0 conda-forge proj 9.0.1 h93bde94_1 conda-forge pthread-stubs 0.4 h36c2ea0_1001 conda-forge pycparser 2.21 pyhd8ed1ab_0 conda-forge pyopenssl 22.0.0 pyhd8ed1ab_0 conda-forge pyparsing 3.0.9 pyhd8ed1ab_0 conda-forge pyproj 3.3.1 py310hf94497c_1 conda-forge pysocks 1.7.1 py310hff52083_5 conda-forge python 3.10.5 h582c2e5_0_cpython conda-forge python-dateutil 2.8.2 pyhd8ed1ab_0 conda-forge python_abi 3.10 2_cp310 conda-forge pytz 2022.1 pyhd8ed1ab_0 conda-forge rasterio 1.3.0 py310h1bedc6d_0 conda-forge readline 8.1.2 h0f457ee_0 conda-forge requests 2.28.1 pyhd8ed1ab_0 conda-forge rtree 1.0.0 py310hbdcdc62_1 conda-forge scikit-learn 1.1.1 py310hffb9edd_0 conda-forge scipy 1.9.0 py310hdfbd76f_0 conda-forge setuptools 63.3.0 py310hff52083_0 conda-forge shapely 1.8.2 py310h5e49deb_3 conda-forge six 1.16.0 pyh6c4a22f_0 conda-forge snappy 1.1.9 hbd366e4_1 conda-forge snuggs 1.4.7 py_0 conda-forge sqlite 3.39.2 h4ff8645_0 conda-forge threadpoolctl 3.1.0 pyh8a188c0_0 conda-forge tiledb 2.9.5 h1e4a385_0 conda-forge tk 8.6.12 h27826a3_0 conda-forge tzcode 2022a h166bdaf_0 conda-forge tzdata 2022a h191b570_0 conda-forge unicodedata2 14.0.0 py310h5764c6d_1 conda-forge urllib3 1.26.11 pyhd8ed1ab_0 conda-forge wheel 0.37.1 pyhd8ed1ab_0 conda-forge xerces-c 3.2.3 h55805fa_5 conda-forge xorg-kbproto 1.0.7 h7f98852_1002 conda-forge xorg-libice 1.0.10 h7f98852_0 conda-forge xorg-libsm 1.2.3 hd9c2040_1000 conda-forge xorg-libx11 1.7.2 h7f98852_0 conda-forge xorg-libxau 1.0.9 h7f98852_0 conda-forge xorg-libxdmcp 1.1.3 h7f98852_0 conda-forge xorg-libxext 1.3.4 h7f98852_1 conda-forge xorg-libxrender 0.9.10 h7f98852_1003 conda-forge xorg-renderproto 0.11.1 h7f98852_1002 conda-forge xorg-xextproto 7.3.0 h7f98852_1002 conda-forge xorg-xproto 7.0.31 h7f98852_1007 conda-forge xyzservices 2022.6.0 pyhd8ed1ab_0 conda-forge xz 5.2.5 h516909a_1 conda-forge zlib 1.2.12 h166bdaf_2 conda-forge zstd 1.5.2 h8a70e8d_2 conda-forge ```

Reproducible example (OSMnx 1.2.1)

import os, zipfile, requests, geopandas as gpd, osmnx as ox
ox.settings.log_console=True
ox.settings.use_cache=True

# point to the shapefile for counties
counties_shapefile_url = 'http://www2.census.gov/geo/tiger/GENZ2016/shp/cb_2016_us_county_500k.zip'

# identify bay area counties by fips code
bayarea = {'Alameda':'001',
           'Contra Costa':'013',
           'Marin':'041',
           'Napa':'055',
           'San Francisco':'075',
           'San Mateo':'081',
           'Santa Clara':'085',
           'Solano':'095',
           'Sonoma':'097'}

# download and read the shapefile
counties_shapefile_zip = counties_shapefile_url[counties_shapefile_url.rfind('/') + 1 :]
counties_shapefile_dir = counties_shapefile_zip[: counties_shapefile_zip.rfind('.zip')]
if not os.path.exists(counties_shapefile_dir):
    response = requests.get(counties_shapefile_url)
    with open(counties_shapefile_zip, 'wb') as f:
        f.write(response.content)
        with zipfile.ZipFile(counties_shapefile_zip, 'r') as zip_file:
            zip_file.extractall(counties_shapefile_dir)
    os.remove(counties_shapefile_zip)

counties = gpd.read_file(counties_shapefile_dir)
len(counties)

# retain only those tracts that are in the bay area counties
mask = (counties['STATEFP'] == '06') & (counties['COUNTYFP'].isin(bayarea.values()))
gdf_bay = counties[mask]
len(gdf_bay)

bayarea_polygon = gdf_bay.unary_union

# get the convex hull, otherwise we'll cut out bridges over the bay
bayarea_polygon = bayarea_polygon.convex_hull

# download the street network & plot
G = ox.graph_from_polygon(bayarea_polygon, network_type='drive', simplify=True)
fig, ax = ox.plot_graph(G)

[MaherSaid]

A temporary get-around for the above issue that I switched to - after figuring out which combination of counties doesn't result in an error - is using nx.compose() to join the multiple networks into one large network. Here's an example using the bay area.

My concern with this is the potential for nodes/edges not connecting properly on the edges, so I'm only using this temporarily until I have a better solution. This is also not very sustainable given the time required for API calls for the different combinations of counties being tested.

[gboeing]

I just ran your snippet without error. Here's my log output (from the second time running it, hence everything is cached):

2022-08-01 15:32:46 Projected GeoDataFrame to +proj=utm +zone=10 +ellps=WGS84 +datum=WGS84 +units=m +no_defs +type=crs
2022-08-01 15:32:46 Projected GeoDataFrame to epsg:4326
2022-08-01 15:32:46 Projected GeoDataFrame to +proj=utm +zone=10 +ellps=WGS84 +datum=WGS84 +units=m +no_defs +type=crs
2022-08-01 15:32:46 Projected GeoDataFrame to epsg:4326
2022-08-01 15:32:46 Requesting data within polygon from API in 21 request(s)
2022-08-01 15:32:46 Retrieved response from cache file "cache/b546a081b3960285cd9322359d5b5beda4a7ec7f.json"
2022-08-01 15:32:46 Retrieved response from cache file "cache/61d94f5874162ffef152b88b43faf02675b43ff2.json"
2022-08-01 15:32:46 Retrieved response from cache file "cache/70de3b142e5dd9720b41f024c8b9319205d6313d.json"
2022-08-01 15:32:46 Retrieved response from cache file "cache/753bb249b47e4fac2706729b986412e99968b1e6.json"
2022-08-01 15:32:46 Retrieved response from cache file "cache/28c6041c9396c1b5333a06a747a79f8498b6e2c6.json"
2022-08-01 15:32:46 Retrieved response from cache file "cache/f363982181e5a6c41d176158e68fecac55d56fe8.json"
2022-08-01 15:32:47 Retrieved response from cache file "cache/4c0efbc6d05d4b5c2e3d6861177c865f3d1fc35c.json"
2022-08-01 15:32:47 Retrieved response from cache file "cache/99e480a1082e178309d09504e426e0677faefcba.json"
2022-08-01 15:32:47 Retrieved response from cache file "cache/6c2ffa7e9b86d5cfe9a8de266d51ee6041b934a6.json"
2022-08-01 15:32:47 Retrieved response from cache file "cache/4b656eb4e0ae0e70e1a58db5d877c07e18581db5.json"
2022-08-01 15:32:47 Retrieved response from cache file "cache/e89168dd95b68c067aedcbea48c1536d09b9cf08.json"
2022-08-01 15:32:48 Retrieved response from cache file "cache/7fa107317ebd6f3fe84a49a038b66cbc807933b5.json"
2022-08-01 15:32:48 Retrieved response from cache file "cache/868368b2d63a7e7528b99f47a2a36dfb05e35e3e.json"
2022-08-01 15:32:48 Retrieved response from cache file "cache/1e6824abfcfc595239852421faa40e190ea8f3b5.json"
2022-08-01 15:32:48 Retrieved response from cache file "cache/45d09f4c40d05d878758bedd4886ea7ea34a20d8.json"
2022-08-01 15:32:48 Retrieved response from cache file "cache/79884905de93bbd713780d9af3c124f001c6cf1c.json"
2022-08-01 15:32:48 Retrieved response from cache file "cache/8d507b17ad23b01f6ba800ec54c247814e388d2c.json"
2022-08-01 15:32:48 Retrieved response from cache file "cache/a41592e1f4b53280ab68d2703c31f30908ee76b7.json"
2022-08-01 15:32:48 Retrieved response from cache file "cache/5722f4579685b0f2629c60d6a9cad2c87b3882f1.json"
2022-08-01 15:32:48 Retrieved response from cache file "cache/672df518e1b908dbd4599c47a6525912301c7ddc.json"
2022-08-01 15:32:48 Retrieved response from cache file "cache/e14e5258c75a032691a9afcf334218f54bdb3749.json"
2022-08-01 15:32:48 Got all network data within polygon from API in 21 request(s)
2022-08-01 15:32:48 Creating graph from downloaded OSM data...
2022-08-01 15:33:02 Created graph with 1501962 nodes and 2799653 edges
2022-08-01 15:33:10 Added length attributes to graph edges
2022-08-01 15:33:10 Identifying all nodes that lie outside the polygon...
2022-08-01 15:33:33 Created nodes GeoDataFrame from graph
2022-08-01 15:33:50 Created r-tree spatial index for 1501962 geometries
2022-08-01 15:33:54 Identified 1490727 geometries inside polygon
2022-08-01 15:34:04 Removed 11235 nodes outside polygon
2022-08-01 15:34:04 Truncated graph by polygon
2022-08-01 15:34:06 Begin topologically simplifying the graph...
2022-08-01 15:34:24 Identified 223012 edge endpoints
2022-08-01 15:34:55 Found 2 edges between 65430220 and 9289642713 when simplifying
2022-08-01 15:35:08 Simplified graph: 1490727 to 223012 nodes, 2777282 to 539976 edges
2022-08-01 15:35:09 Identifying all nodes that lie outside the polygon...
2022-08-01 15:35:12 Created nodes GeoDataFrame from graph
2022-08-01 15:35:14 Created r-tree spatial index for 223012 geometries
2022-08-01 15:35:15 Identified 221995 geometries inside polygon
2022-08-01 15:35:18 Removed 1017 nodes outside polygon
2022-08-01 15:35:21 Removed 34 isolated nodes
2022-08-01 15:35:26 Got largest weakly connected component (219633 of 221961 total nodes)
2022-08-01 15:35:26 Truncated graph by polygon
2022-08-01 15:35:27 Counted undirected street segments incident on each node
2022-08-01 15:35:27 graph_from_polygon returned graph with 219633 nodes and 534196 edges

This is on a Mac with Python 3.10 and OSMnx 1.2.1.

[MaherSaid]

Thank you for the response. Indeed, I tested it on my personal computer and it ran fine. Same code, same conda requirements (OSMnx 1.2.1 on Windows 11 21H2 build 22000.795). I also tested it on another machine with success (OSMnx 1.2.1 on Windows 10 21H1 build 19043.1766).

However, on the problematic machine, trying with a fresh conda environment (again, with the same requirements) still results in the same issue. This occurs even when using the same cache from the successful runs on the other machines. It runs fine in a docker on this same machine.

Let me know if you want me to try anything or to provide any additional info. Otherwise, feel free to close this issue if you think it's an isolated case.

[gboeing]

I'll have access to my Ubuntu machine again in a few days. We can leave this issue open until I test it on that machine, in case there's a linux-related issue. However, even if there is, it sounds like it would be related to some dependency in the stack rather than OSMnx itself.

[gboeing]

I'm unable to reproduce this on Ubuntu 22.04 either. I'm going to close this as it seems to be an isolated case that I cannot reproduce.