search

InseeFrLab / cartiflette

Projet visant à simplifier la récupération des shapefiles officiels
https://inseefrlab.github.io/cartiflette-website/
MIT License
46 stars 6 forks source link

Explorer un provider avec l'API carto (en version beta sur les flux du géoportail) #57

Open tgrandje opened 1 year ago

tgrandje commented 1 year ago

Pas réussi à attaquer directement les flux WFS (retour systématique d'un "MissingParameter" non précisé).

Ne fonctionne pas :

from owslib.wfs import WebFeatureService
url = "https://wxs.ign.fr/geoportail/wfs"
wfs = WebFeatureService(url=url, version='2.0.0')
cog_files = [
    x for x in wfs.contents
    if 'cog' in x.lower()
    and 'carto' in x.lower()
    and 'latest' in x.lower()
    ]
print(wfs.getfeature(cog_files[-1]).read())

En revanche, l'appel via l'API carto en version beta fonctionne ; le débit semble beaucoup plus rapide que sur le site de l'IGN (mais les md5 ne sont par contre pas fournis, il faudra les calculer à réception) :

from owslib.wfs import WebFeatureService
import geopandas as gpd
import requests

url = "https://wxs.ign.fr/geoportail/wfs"
wfs = WebFeatureService(url=url, version='2.0.0')
cog_files = [
    x for x in wfs.contents
    if 'cog' in x.lower()
    and 'carto' in x.lower()
    and 'latest' in x.lower()
    ]
print(cog_files)
file = cog_files[-1]
bbox_world = """{"type":"Polygon", "coordinates":[[[-180,-90],[-180,90],[180,90],[180,-90],[-180,-90]]]}"""
r = requests.get(
    'https://apicarto.ign.fr/api/wfs-geoportail/search',
    params={"source":file, "geom":bbox_world}
    )
gdf = gpd.GeoDataFrame.from_features(r.json())
gdf.plot()
tgrandje commented 1 year ago

Poursuite des tests : résultats de l'API plafonnés par sections de 1000 entités, ce qui ralentit l'opération.

Code complet avec boucle :

bbox_world = """{"type":"Polygon", "coordinates":[[[-180,-90],[-180,90],[180,90],[180,-90],[-180,-90]]]}"""
r = requests.get(
    'https://apicarto.ign.fr/api/wfs-geoportail/search',
    params={"source":"ADMINEXPRESS-COG-CARTO.LATEST:commune", "geom":bbox_world},
    stream=True
    )
data = []
while True:
    chunk = r.json()
    data.append(gpd.GeoDataFrame.from_features(chunk))
    try:
        link = [x for x in chunk["links"] if x['rel'] == "next"][0]['href']
    except Exception:
        break
    r = requests.get(link, stream=True)
gdf = gpd.pd.concat(data)

Résultats :

print(gdf.shape)
>>>  (34955, 14)

et plus en détails :

print(gdf.head())
>>>
                                             geometry  \
0  MULTIPOLYGON (((3.61424 45.27211, 3.61381 45.2...   
1  MULTIPOLYGON (((-0.02484 43.43001, -0.02712 43...   
2  MULTIPOLYGON (((-3.97907 47.70396, -3.97903 47...   
3  MULTIPOLYGON (((-3.20269 48.84901, -3.20227 48...   
4  MULTIPOLYGON (((5.43224 45.61478, 5.43227 45.6...   

                         id           nom         nom_m insee_com  \
0  COMMUNE_0000000009754033    Connangles    CONNANGLES     43076   
1  COMMUNE_0000000009760784       Vidouze       VIDOUZE     65462   
2  COMMUNE_0000000009742077     Fouesnant     FOUESNANT     29058   
3  COMMUNE_0000000009735245  Plougrescant  PLOUGRESCANT     22218   
4  COMMUNE_0000000009752504     Montcarra     MONTCARRA     38250   

           statut  population insee_can insee_arr insee_dep insee_reg  \
0  Commune simple         137        11         1        43        84   
1  Commune simple         243        13         3        65        76   
2  Commune simple        9864        11         4        29        53   
3  Commune simple        1166        27         3        22        53   
4  Commune simple         569        24         2        38        84   

  siren_epci                                               bbox code_insee  
0  200073419  [3.60602357, 45.26946656, 3.69063473, 45.32823...       None  
1  200072106  [-0.0707617, 43.40933897, -0.01644636, 43.4660...       None  
2  242900660  [-4.08875845, 47.70128144, -3.95087428, 47.927...       None  
3  200065928  [-3.27502707, 48.80900609, -3.1798906, 48.8756...       None  
4  200068542  [5.36914541, 45.60242001, 5.43423945, 45.61578...       None
tgrandje commented 1 year ago

Liste exhaustive des fichiers COG disponibles sur le flux :

 'ADMINEXPRESS-COG-CARTO.LATEST:arrondissement',
 'ADMINEXPRESS-COG-CARTO.LATEST:arrondissement_municipal',
 'ADMINEXPRESS-COG-CARTO.LATEST:canton',
 'ADMINEXPRESS-COG-CARTO.LATEST:chflieu_arrondissement_municipal',
 'ADMINEXPRESS-COG-CARTO.LATEST:chflieu_commune',
 'ADMINEXPRESS-COG-CARTO.LATEST:chflieu_commune_associee_ou_deleguee',
 'ADMINEXPRESS-COG-CARTO.LATEST:collectivite_territoriale',
 'ADMINEXPRESS-COG-CARTO.LATEST:commune',
 'ADMINEXPRESS-COG-CARTO.LATEST:commune_associee_ou_deleguee',
 'ADMINEXPRESS-COG-CARTO.LATEST:departement',
 'ADMINEXPRESS-COG-CARTO.LATEST:epci',
 'ADMINEXPRESS-COG-CARTO.LATEST:region',
 'ADMINEXPRESS-COG-CARTO_BDD_WLD_WM_04-2022:arrondissement',
 'ADMINEXPRESS-COG-CARTO_BDD_WLD_WM_04-2022:arrondissement_municipal',
 'ADMINEXPRESS-COG-CARTO_BDD_WLD_WM_04-2022:canton',
 'ADMINEXPRESS-COG-CARTO_BDD_WLD_WM_04-2022:chflieu_arrondissement_municipal',
 'ADMINEXPRESS-COG-CARTO_BDD_WLD_WM_04-2022:chflieu_commune',
 'ADMINEXPRESS-COG-CARTO_BDD_WLD_WM_04-2022:chflieu_commune_associee_ou_deleguee',
 'ADMINEXPRESS-COG-CARTO_BDD_WLD_WM_04-2022:collectivite_territoriale',
 'ADMINEXPRESS-COG-CARTO_BDD_WLD_WM_04-2022:commune',
 'ADMINEXPRESS-COG-CARTO_BDD_WLD_WM_04-2022:commune_associee_ou_deleguee',
 'ADMINEXPRESS-COG-CARTO_BDD_WLD_WM_04-2022:departement',
 'ADMINEXPRESS-COG-CARTO_BDD_WLD_WM_04-2022:epci',
 'ADMINEXPRESS-COG-CARTO_BDD_WLD_WM_04-2022:region',
 'ADMINEXPRESS-COG-CARTO_BDD_WLD_WM_05-2022:arrondissement',
 'ADMINEXPRESS-COG-CARTO_BDD_WLD_WM_05-2022:arrondissement_municipal',
 'ADMINEXPRESS-COG-CARTO_BDD_WLD_WM_05-2022:canton',
 'ADMINEXPRESS-COG-CARTO_BDD_WLD_WM_05-2022:chflieu_arrondissement_municipal',
 'ADMINEXPRESS-COG-CARTO_BDD_WLD_WM_05-2022:chflieu_commune',
 'ADMINEXPRESS-COG-CARTO_BDD_WLD_WM_05-2022:chflieu_commune_associee_ou_deleguee',
 'ADMINEXPRESS-COG-CARTO_BDD_WLD_WM_05-2022:collectivite_territoriale',
 'ADMINEXPRESS-COG-CARTO_BDD_WLD_WM_05-2022:commune',
 'ADMINEXPRESS-COG-CARTO_BDD_WLD_WM_05-2022:commune_associee_ou_deleguee',
 'ADMINEXPRESS-COG-CARTO_BDD_WLD_WM_05-2022:departement',
 'ADMINEXPRESS-COG-CARTO_BDD_WLD_WM_05-2022:epci',
 'ADMINEXPRESS-COG-CARTO_BDD_WLD_WM_05-2022:region',
 'ADMINEXPRESS-COG-CARTO_BDD_WLD_WM_06-2021:arrondissement',
 'ADMINEXPRESS-COG-CARTO_BDD_WLD_WM_06-2021:arrondissement_municipal',
 'ADMINEXPRESS-COG-CARTO_BDD_WLD_WM_06-2021:canton',
 'ADMINEXPRESS-COG-CARTO_BDD_WLD_WM_06-2021:chflieu_arrondissement_municipal',
 'ADMINEXPRESS-COG-CARTO_BDD_WLD_WM_06-2021:chflieu_commune',
 'ADMINEXPRESS-COG-CARTO_BDD_WLD_WM_06-2021:chflieu_commune_associee_ou_deleguee',
 'ADMINEXPRESS-COG-CARTO_BDD_WLD_WM_06-2021:collectivite_territoriale',
 'ADMINEXPRESS-COG-CARTO_BDD_WLD_WM_06-2021:commune',
 'ADMINEXPRESS-COG-CARTO_BDD_WLD_WM_06-2021:commune_associee_ou_deleguee',
 'ADMINEXPRESS-COG-CARTO_BDD_WLD_WM_06-2021:departement',
 'ADMINEXPRESS-COG-CARTO_BDD_WLD_WM_06-2021:epci',
 'ADMINEXPRESS-COG-CARTO_BDD_WLD_WM_06-2021:region',
 'ADMINEXPRESS-COG-CARTO_BDD_WLD_WM_2021:arrondissement',
 'ADMINEXPRESS-COG-CARTO_BDD_WLD_WM_2021:arrondissement_municipal',
 'ADMINEXPRESS-COG-CARTO_BDD_WLD_WM_2021:canton',
 'ADMINEXPRESS-COG-CARTO_BDD_WLD_WM_2021:chflieu_arrondissement_municipal',
 'ADMINEXPRESS-COG-CARTO_BDD_WLD_WM_2021:chflieu_commune',
 'ADMINEXPRESS-COG-CARTO_BDD_WLD_WM_2021:chflieu_commune_associee_ou_deleguee',
 'ADMINEXPRESS-COG-CARTO_BDD_WLD_WM_2021:collectivite_territoriale',
 'ADMINEXPRESS-COG-CARTO_BDD_WLD_WM_2021:commune',
 'ADMINEXPRESS-COG-CARTO_BDD_WLD_WM_2021:commune_associee_ou_deleguee',
 'ADMINEXPRESS-COG-CARTO_BDD_WLD_WM_2021:departement',
 'ADMINEXPRESS-COG-CARTO_BDD_WLD_WM_2021:epci',
 'ADMINEXPRESS-COG-CARTO_BDD_WLD_WM_2021:region',
 'ADMINEXPRESS_COG_2017_CARTO:arrondissement_departemental_carto',
 'ADMINEXPRESS_COG_2017_CARTO:commune_carto',
 'ADMINEXPRESS_COG_2017_CARTO:departement_carto',
 'ADMINEXPRESS_COG_2017_CARTO:epci_carto',
 'ADMINEXPRESS_COG_2017_CARTO:region_carto',
 'ADMINEXPRESS_COG_2017_CARTO_BDD_WLD_WM_20170708:arrondissement_departemental_carto',
 'ADMINEXPRESS_COG_2017_CARTO_BDD_WLD_WM_20170708:commune_carto',
 'ADMINEXPRESS_COG_2017_CARTO_BDD_WLD_WM_20170708:departement_carto',
 'ADMINEXPRESS_COG_2017_CARTO_BDD_WLD_WM_20170708:epci_carto',
 'ADMINEXPRESS_COG_2017_CARTO_BDD_WLD_WM_20170708:region_carto',
 'ADMINEXPRESS_COG_2017_CARTO_BDD_WLD_WM_20170709:arrondissement_departemental_carto',
 'ADMINEXPRESS_COG_2017_CARTO_BDD_WLD_WM_20170709:commune_carto',
 'ADMINEXPRESS_COG_2017_CARTO_BDD_WLD_WM_20170709:departement_carto',
 'ADMINEXPRESS_COG_2017_CARTO_BDD_WLD_WM_20170709:epci_carto',
 'ADMINEXPRESS_COG_2017_CARTO_BDD_WLD_WM_20170709:region_carto',
 'ADMINEXPRESS_COG_2018_CARTO:arrondissement_departemental_carto',
 'ADMINEXPRESS_COG_2018_CARTO:commune_carto',
 'ADMINEXPRESS_COG_2018_CARTO:departement_carto',
 'ADMINEXPRESS_COG_2018_CARTO:epci_carto',
 'ADMINEXPRESS_COG_2018_CARTO:region_carto',
 'ADMINEXPRESS_COG_2018_CARTO_BDD_WLD_WM_20180515:arrondissement_departemental_carto',
 'ADMINEXPRESS_COG_2018_CARTO_BDD_WLD_WM_20180515:commune_carto',
 'ADMINEXPRESS_COG_2018_CARTO_BDD_WLD_WM_20180515:departement_carto',
 'ADMINEXPRESS_COG_2018_CARTO_BDD_WLD_WM_20180515:epci_carto',
 'ADMINEXPRESS_COG_2018_CARTO_BDD_WLD_WM_20180515:region_carto',
 'ADMINEXPRESS_COG_2019_CARTO:arrondissement_departemental_carto',
 'ADMINEXPRESS_COG_2019_CARTO:chef_lieu_carto',
 'ADMINEXPRESS_COG_2019_CARTO:commune_carto',
 'ADMINEXPRESS_COG_2019_CARTO:departement_carto',
 'ADMINEXPRESS_COG_2019_CARTO:epci_carto',
 'ADMINEXPRESS_COG_2019_CARTO:region_carto',
 'ADMINEXPRESS_COG_2019_CARTO_BDD_WLD_WM_20190924:arrondissement_departemental_carto',
 'ADMINEXPRESS_COG_2019_CARTO_BDD_WLD_WM_20190924:chef_lieu_carto',
 'ADMINEXPRESS_COG_2019_CARTO_BDD_WLD_WM_20190924:commune_carto',
 'ADMINEXPRESS_COG_2019_CARTO_BDD_WLD_WM_20190924:departement_carto',
 'ADMINEXPRESS_COG_2019_CARTO_BDD_WLD_WM_20190924:epci_carto',
 'ADMINEXPRESS_COG_2019_CARTO_BDD_WLD_WM_20190924:region_carto',
 'ADMINEXPRESS_COG_2020_CARTO:arrondissement_departemental_carto',
 'ADMINEXPRESS_COG_2020_CARTO:canton_carto',
 'ADMINEXPRESS_COG_2020_CARTO:chef_lieu_carto',
 'ADMINEXPRESS_COG_2020_CARTO:commune_carto',
 'ADMINEXPRESS_COG_2020_CARTO:departement_carto',
 'ADMINEXPRESS_COG_2020_CARTO:epci_carto',
 'ADMINEXPRESS_COG_2020_CARTO:region_carto',
 'ADMINEXPRESS_COG_2020_CARTO_BDD_WLD_WM_17-09-2020:arrondissement_departemental_carto',
 'ADMINEXPRESS_COG_2020_CARTO_BDD_WLD_WM_17-09-2020:canton_carto',
 'ADMINEXPRESS_COG_2020_CARTO_BDD_WLD_WM_17-09-2020:chef_lieu_carto',
 'ADMINEXPRESS_COG_2020_CARTO_BDD_WLD_WM_17-09-2020:commune_carto',
 'ADMINEXPRESS_COG_2020_CARTO_BDD_WLD_WM_17-09-2020:departement_carto',
 'ADMINEXPRESS_COG_2020_CARTO_BDD_WLD_WM_17-09-2020:epci_carto',
 'ADMINEXPRESS_COG_2020_CARTO_BDD_WLD_WM_17-09-2020:region_carto',
 'ADMINEXPRESS_COG_2020_CARTO_BDD_WLD_WM_18-12-2020:arrondissement_departemental_carto',
 'ADMINEXPRESS_COG_2020_CARTO_BDD_WLD_WM_18-12-2020:canton_carto',
 'ADMINEXPRESS_COG_2020_CARTO_BDD_WLD_WM_18-12-2020:chef_lieu_carto',
 'ADMINEXPRESS_COG_2020_CARTO_BDD_WLD_WM_18-12-2020:commune_carto',
 'ADMINEXPRESS_COG_2020_CARTO_BDD_WLD_WM_18-12-2020:departement_carto',
 'ADMINEXPRESS_COG_2020_CARTO_BDD_WLD_WM_18-12-2020:epci_carto',
 'ADMINEXPRESS_COG_2020_CARTO_BDD_WLD_WM_18-12-2020:region_carto',
 'ADMINEXPRESS_COG_2020_CARTO_BDD_WLD_WM_WFS:arrondissement_departemental_carto',
 'ADMINEXPRESS_COG_2020_CARTO_BDD_WLD_WM_WFS:canton_carto',
 'ADMINEXPRESS_COG_2020_CARTO_BDD_WLD_WM_WFS:chef_lieu_carto',
 'ADMINEXPRESS_COG_2020_CARTO_BDD_WLD_WM_WFS:commune_carto',
 'ADMINEXPRESS_COG_2020_CARTO_BDD_WLD_WM_WFS:departement_carto',
 'ADMINEXPRESS_COG_2020_CARTO_BDD_WLD_WM_WFS:epci_carto',
 'ADMINEXPRESS_COG_2020_CARTO_BDD_WLD_WM_WFS:region_carto',
 'ADMINEXPRESS_COG_2020_CARTO_BDD_WLD_WM_WFS_07-2020:arrondissement_departemental_carto',
 'ADMINEXPRESS_COG_2020_CARTO_BDD_WLD_WM_WFS_07-2020:canton_carto',
 'ADMINEXPRESS_COG_2020_CARTO_BDD_WLD_WM_WFS_07-2020:chef_lieu_carto',
 'ADMINEXPRESS_COG_2020_CARTO_BDD_WLD_WM_WFS_07-2020:commune_carto',
 'ADMINEXPRESS_COG_2020_CARTO_BDD_WLD_WM_WFS_07-2020:departement_carto',
 'ADMINEXPRESS_COG_2020_CARTO_BDD_WLD_WM_WFS_07-2020:epci_carto',
 'ADMINEXPRESS_COG_2020_CARTO_BDD_WLD_WM_WFS_07-2020:region_carto'