Wondering how to adapt https://github.com/umap-project/umap/issues/112 to other data sources like data gouv and grist

[virgile-dev]

We've been discussing with Yohan ideas around adding dedicated UIs so that users can easily import data from Data Gouv, Grist and OpenDataSoft. Here is what he wrote a little while ago.

• Aujourd’hui uMap est très utilisé pour ajouter des informations à la main ou importer des données depuis des fichiers, mais la plupart des utilisateurs cibles de uMap ne savent pas où trouver de la donnée, quel format utiliser, etc. Un des effets de bord est que souvent la donnée est recréée à la main. Dans la mission de uMap de permettre à des utilisateurs non techniciens de créer des carte, cette fonctionnalité nous semble être une évolution naturelle.
• En termes UX, on imagine au moment de l’import de données (ou de l’utilisation de données distantes) d’avoir, en plus de la sélection d’un fichier, d’une URL ou d’un champ de copier-coller, d’avoir un sélecteur, par exemple “Importer depuis data.gouv.fr”, “Importer depuis OpenStreetMap”, qui ensuite ouvre un formulaire permettant, soit de naviguer dans des jeux de données présélectionnés, soit de faire une recherche libre dans les jeux de donnée
• pouvoir typer les données d’un calque: uMap est souvent utilisé pour faire de la collecte, mais aujourd’hui tous les champs sont des chaînes de caractère; en pouvant typer les champs (nombre, date, et pourquoi pas code commune, numéro siret, etc.) on pourrait permettre d’avoir une donnée collectée de meilleure qualité et donc potentiellement publiable en open data
• permettre des backends d’écriture autres que uMap, par exemple Grist, Google Sheet, ou pourquoi pas data.gouv.fr aussi: au lieu d’écrire la donnée localement, on l’écrit sur un serveur distant qui apporte d’autres fonctionnalités, qui sont en dehors du scope de uMap (par exemple la validation des données, ou l’exploitation des données en tableau, etc.). Ceci ferait de uMap une sorte de “client d’édition de données géographiques agnostiques”, qui pourrait soit marcher en stand alone, soit se brancher sur d’autres applications. C’est plus ou moins une proposition alternative à la proposition précédente (typer les données)
• Faciliter la visualisation de données dans uMap: aujourd’hui il possible d’ouvrir un jeu de données dans uMap (c’était historiquement disponible dans data.gouv.fr, c’est par exemple utilisé dans deveco) de la sorte: https://monserveurumap.fr/map/new/?dataUrl=url-des-données&dataFormat=format-des-données
• On pourrait améliorer deux points:
  • pouvoir aussi ajouter une feuille de style (en json) en paramètre
  • atterir sur une carte en visualisation plutôt qu’en édition directe, sinon parfois ça peut prếter à confusion (est-ce que je modifie les données sources quand je modifie un point ?), donc arriver sur une vue de carte sans les boutons d’édition mais avec un bouton du type “Créer une carte personnalisée avec ces données”

[virgile-dev]

A more bite size issue (use case oriented) can be found here : https://github.com/umap-project/umap-dsfr/issues/15

[davidbgk]

The hard thing is that we'll probably have to create/deploy some kind of proxy à-la-geodatamine to handle ressources in the correct/trimmed format from datagouv. For instance contours géographiques with a reasonable definition are quite heavy so we need to split the file and only download the pertinent part for a city, etc.

[virgile-dev]

I find it surprising that there are no ready to use APIs from data gouv or etalab that we could use to retreive a specific geojson object. If it doesn't exist inside DINUM maybe we should ask for a ftap data to build something like this.

[davidbgk]

There is already a GeoAPI but without contours as far as I know.

[virgile-dev]

Thanks ! Actually yes it does seem to provide provide the contour. With this command : curl 'https://geo.api.gouv.fr/communes?nom=Versailles&fields=code,nom,contour' I get this result

[
  {
    "code": "78646",
    "nom": "Versailles",
    "contour": {
      "type": "Polygon",
      "coordinates": [
        [
          [
            2.105717,
            48.780691
          ],
          [
            2.104601,
            48.780417
          ],
          [
            2.103279,
            48.780051
          ],
          [
            2.103204,
            48.780048
          ],
          [
            2.10234,
            48.780293
          ],
          [
            2.101322,
            48.780514
          ],
          [
            2.100149,
            48.780747
          ],
          [
            2.099209,
            48.780911
          ],
          [
            2.096207,
            48.781483
          ],
          [
            2.093502,
            48.782008
          ],
          [
            2.091268,
            48.782435
          ],
          [
            2.091037,
            48.782494
          ],
          [
            2.089366,
            48.783073
          ],
          [
            2.08902,
            48.783204
          ],
          [
            2.089002,
            48.783267
          ],
          [
            2.088565,
            48.783425
          ],
          [
            2.087176,
            48.783823
          ],
          [
            2.086857,
            48.783955
          ],
          [
            2.085278,
            48.784543
          ],
          [
            2.084875,
            48.784632
          ],
          [
            2.084528,
            48.784769
          ],
          [
            2.084251,
            48.784962
          ],
          [
            2.084114,
            48.785316
          ],
          [
            2.083991,
            48.785813
          ],
          [
            2.083581,
            48.786781
          ],
          [
            2.083452,
            48.787106
          ],
          [
            2.082985,
            48.788542
          ],
          [
            2.080235,
            48.78897
          ],
          [
            2.077758,
            48.7893
          ],
          [
            2.076081,
            48.788772
          ],
          [
            2.074703,
            48.788661
          ],
          [
            2.074414,
            48.788657
          ],
          [
            2.069909,
            48.789016
          ],
          [
            2.069955,
            48.789236
          ],
          [
            2.070013,
            48.789283
          ],
          [
            2.070144,
            48.789869
          ],
          [
            2.070502,
            48.790636
          ],
          [
            2.070588,
            48.790989
          ],
          [
            2.070747,
            48.791334
          ],
          [
            2.070996,
            48.792045
          ],
          [
            2.071699,
            48.793923
          ],
          [
            2.071887,
            48.794341
          ],
          [
            2.07193,
            48.794529
          ],
          [
            2.072618,
            48.796405
          ],
          [
            2.07285,
            48.797018
          ],
          [
            2.072995,
            48.797455
          ],
          [
            2.073198,
            48.798272
          ],
          [
            2.073277,
            48.798496
          ],
          [
            2.074185,
            48.800583
          ],
          [
            2.074076,
            48.800871
          ],
          [
            2.074089,
            48.801047
          ],
          [
            2.074386,
            48.801049
          ],
          [
            2.075039,
            48.802984
          ],
          [
            2.07549,
            48.804239
          ],
          [
            2.076094,
            48.805862
          ],
          [
            2.076283,
            48.806433
          ],
          [
            2.077083,
            48.808463
          ],
          [
            2.077903,
            48.810762
          ],
          [
            2.078341,
            48.812238
          ],
          [
            2.078721,
            48.813223
          ],
          [
            2.07913,
            48.814347
          ],
          [
            2.079432,
            48.815219
          ],
          [
            2.079859,
            48.815909
          ],
          [
            2.08095,
            48.816567
          ],
          [
            2.082054,
            48.817242
          ],
          [
            2.082225,
            48.817356
          ],
          [
            2.082241,
            48.817541
          ],
          [
            2.082583,
            48.818123
          ],
          [
            2.083038,
            48.818761
          ],
          [
            2.083488,
            48.81934
          ],
          [
            2.083858,
            48.819697
          ],
          [
            2.083924,
            48.820698
          ],
          [
            2.084578,
            48.820688
          ],
          [
            2.085366,
            48.821686
          ],
          [
            2.08713,
            48.823906
          ],
          [
            2.088905,
            48.826071
          ],
          [
            2.088704,
            48.826276
          ],
          [
            2.089142,
            48.826481
          ],
          [
            2.089179,
            48.826527
          ],
          [
            2.089456,
            48.826355
          ],
          [
            2.090668,
            48.8252
          ],
          [
            2.09245,
            48.823452
          ],
          [
            2.092653,
            48.823567
          ],
          [
            2.095671,
            48.825414
          ],
          [
            2.095745,
            48.825431
          ],
          [
            2.095993,
            48.825345
          ],
          [
            2.102273,
            48.823454
          ],
          [
            2.102462,
            48.823395
          ],
          [
            2.103434,
            48.823898
          ],
          [
            2.103874,
            48.824111
          ],
          [
            2.10455,
            48.824484
          ],
          [
            2.105452,
            48.825802
          ],
          [
            2.105838,
            48.826036
          ],
          [
            2.105991,
            48.8261
          ],
          [
            2.107291,
            48.826867
          ],
          [
            2.108344,
            48.827222
          ],
          [
            2.111007,
            48.82808
          ],
          [
            2.112608,
            48.828578
          ],
          [
            2.113068,
            48.826676
          ],
          [
            2.11324,
            48.826006
          ],
          [
            2.113961,
            48.823051
          ],
          [
            2.114003,
            48.822825
          ],
          [
            2.114111,
            48.822465
          ],
          [
            2.114509,
            48.820787
          ],
          [
            2.114267,
            48.820747
          ],
          [
            2.114089,
            48.820635
          ],
          [
            2.114986,
            48.820411
          ],
          [
            2.115776,
            48.820138
          ],
          [
            2.117842,
            48.819381
          ],
          [
            2.121108,
            48.818247
          ],
          [
            2.124676,
            48.816928
          ],
          [
            2.124947,
            48.816837
          ],
          [
            2.125104,
            48.81675
          ],
          [
            2.128394,
            48.81555
          ],
          [
            2.128488,
            48.815503
          ],
          [
            2.128702,
            48.815584
          ],
          [
            2.128959,
            48.8156
          ],
          [
            2.12911,
            48.81589
          ],
          [
            2.129453,
            48.815804
          ],
          [
            2.129291,
            48.815503
          ],
          [
            2.129485,
            48.81529
          ],
          [
            2.129447,
            48.815036
          ],
          [
            2.13103,
            48.81478
          ],
          [
            2.131731,
            48.814672
          ],
          [
            2.132396,
            48.814557
          ],
          [
            2.136068,
            48.813975
          ],
          [
            2.13626,
            48.81505
          ],
          [
            2.136426,
            48.816027
          ],
          [
            2.136835,
            48.81832
          ],
          [
            2.136899,
            48.818496
          ],
          [
            2.137161,
            48.81872
          ],
          [
            2.137723,
            48.818925
          ],
          [
            2.137846,
            48.819017
          ],
          [
            2.137884,
            48.81913
          ],
          [
            2.137948,
            48.820954
          ],
          [
            2.138001,
            48.821102
          ],
          [
            2.138641,
            48.822037
          ],
          [
            2.139591,
            48.823408
          ],
          [
            2.139676,
            48.823553
          ],
          [
            2.140157,
            48.824241
          ],
          [
            2.142446,
            48.825366
          ],
          [
            2.142752,
            48.825509
          ],
          [
            2.142958,
            48.825674
          ],
          [
            2.144399,
            48.826893
          ],
          [
            2.144486,
            48.826976
          ],
          [
            2.144955,
            48.827154
          ],
          [
            2.145434,
            48.827355
          ],
          [
            2.145884,
            48.827571
          ],
          [
            2.146527,
            48.827971
          ],
          [
            2.146987,
            48.828165
          ],
          [
            2.147622,
            48.828343
          ],
          [
            2.148118,
            48.828455
          ],
          [
            2.148481,
            48.828486
          ],
          [
            2.148762,
            48.827728
          ],
          [
            2.149019,
            48.826959
          ],
          [
            2.150447,
            48.823668
          ],
          [
            2.150961,
            48.822608
          ],
          [
            2.15147,
            48.821406
          ],
          [
            2.151421,
            48.821131
          ],
          [
            2.151244,
            48.820702
          ],
          [
            2.150786,
            48.819869
          ],
          [
            2.150585,
            48.819446
          ],
          [
            2.150302,
            48.81875
          ],
          [
            2.150747,
            48.818844
          ],
          [
            2.150757,
            48.818233
          ],
          [
            2.15079,
            48.818051
          ],
          [
            2.15088,
            48.817906
          ],
          [
            2.151226,
            48.817639
          ],
          [
            2.151402,
            48.817284
          ],
          [
            2.15164,
            48.816974
          ],
          [
            2.152121,
            48.816706
          ],
          [
            2.152627,
            48.816463
          ],
          [
            2.153239,
            48.816139
          ],
          [
            2.153418,
            48.816095
          ],
          [
            2.154059,
            48.816124
          ],
          [
            2.154304,
            48.816121
          ],
          [
            2.154493,
            48.816061
          ],
          [
            2.155676,
            48.815419
          ],
          [
            2.15691,
            48.81481
          ],
          [
            2.157344,
            48.814565
          ],
          [
            2.158543,
            48.813961
          ],
          [
            2.159861,
            48.813488
          ],
          [
            2.160261,
            48.813242
          ],
          [
            2.160491,
            48.813075
          ],
          [
            2.160907,
            48.812904
          ],
          [
            2.161278,
            48.812782
          ],
          [
            2.161212,
            48.812474
          ],
          [
            2.160658,
            48.811747
          ],
          [
            2.160247,
            48.811489
          ],
          [
            2.159473,
            48.81095
          ],
          [
            2.158845,
            48.810545
          ],
          [
            2.158077,
            48.810119
          ],
          [
            2.158436,
            48.809893
          ],
          [
            2.158306,
            48.809743
          ],
          [
            2.158711,
            48.809461
          ],
          [
            2.158616,
            48.809376
          ],
          [
            2.158856,
            48.809179
          ],
          [
            2.159027,
            48.808702
          ],
          [
            2.158975,
            48.808512
          ],
          [
            2.159276,
            48.808342
          ],
          [
            2.159389,
            48.808193
          ],
          [
            2.159875,
            48.807874
          ],
          [
            2.160175,
            48.807645
          ],
          [
            2.160443,
            48.807381
          ],
          [
            2.161211,
            48.806492
          ],
          [
            2.160925,
            48.806502
          ],
          [
            2.161466,
            48.80598
          ],
          [
            2.161782,
            48.806125
          ],
          [
            2.162052,
            48.805792
          ],
          [
            2.161766,
            48.805693
          ],
          [
            2.162242,
            48.805211
          ],
          [
            2.16261,
            48.80487
          ],
          [
            2.163593,
            48.804085
          ],
          [
            2.163761,
            48.803973
          ],
          [
            2.163151,
            48.803394
          ],
          [
            2.162209,
            48.802469
          ],
          [
            2.161962,
            48.802244
          ],
          [
            2.161251,
            48.801546
          ],
          [
            2.160392,
            48.800646
          ],
          [
            2.159202,
            48.799433
          ],
          [
            2.158833,
            48.799041
          ],
          [
            2.159022,
            48.798866
          ],
          [
            2.159212,
            48.798627
          ],
          [
            2.162004,
            48.798732
          ],
          [
            2.162124,
            48.798506
          ],
          [
            2.163193,
            48.796937
          ],
          [
            2.163646,
            48.796237
          ],
          [
            2.163905,
            48.794168
          ],
          [
            2.164391,
            48.794007
          ],
          [
            2.164713,
            48.79385
          ],
          [
            2.164946,
            48.793575
          ],
          [
            2.165582,
            48.792973
          ],
          [
            2.165992,
            48.792692
          ],
          [
            2.166246,
            48.792482
          ],
          [
            2.166397,
            48.792305
          ],
          [
            2.166802,
            48.791751
          ],
          [
            2.166968,
            48.791654
          ],
          [
            2.167734,
            48.791364
          ],
          [
            2.16812,
            48.791252
          ],
          [
            2.168369,
            48.791152
          ],
          [
            2.168389,
            48.79103
          ],
          [
            2.168256,
            48.790893
          ],
          [
            2.167826,
            48.790533
          ],
          [
            2.167531,
            48.790179
          ],
          [
            2.167461,
            48.789971
          ],
          [
            2.167449,
            48.789569
          ],
          [
            2.167532,
            48.789333
          ],
          [
            2.167783,
            48.788964
          ],
          [
            2.168179,
            48.788441
          ],
          [
            2.168335,
            48.788181
          ],
          [
            2.168226,
            48.787966
          ],
          [
            2.168127,
            48.787894
          ],
          [
            2.167593,
            48.787729
          ],
          [
            2.167411,
            48.787712
          ],
          [
            2.166982,
            48.787559
          ],
          [
            2.166808,
            48.787455
          ],
          [
            2.166684,
            48.787305
          ],
          [
            2.16639,
            48.787072
          ],
          [
            2.166245,
            48.78702
          ],
          [
            2.16542,
            48.786868
          ],
          [
            2.16503,
            48.786718
          ],
          [
            2.164843,
            48.786567
          ],
          [
            2.164431,
            48.785927
          ],
          [
            2.164385,
            48.785766
          ],
          [
            2.164465,
            48.785518
          ],
          [
            2.164456,
            48.785385
          ],
          [
            2.163671,
            48.784058
          ],
          [
            2.163448,
            48.783765
          ],
          [
            2.163061,
            48.78338
          ],
          [
            2.162047,
            48.782441
          ],
          [
            2.161967,
            48.782278
          ],
          [
            2.161273,
            48.782472
          ],
          [
            2.160556,
            48.782707
          ],
          [
            2.159655,
            48.783059
          ],
          [
            2.156537,
            48.784467
          ],
          [
            2.154603,
            48.785345
          ],
          [
            2.154316,
            48.785495
          ],
          [
            2.153814,
            48.785705
          ],
          [
            2.151759,
            48.786487
          ],
          [
            2.151426,
            48.786748
          ],
          [
            2.148333,
            48.7872
          ],
          [
            2.148268,
            48.787423
          ],
          [
            2.147852,
            48.789021
          ],
          [
            2.147747,
            48.789478
          ],
          [
            2.147799,
            48.789586
          ],
          [
            2.147051,
            48.790285
          ],
          [
            2.146776,
            48.790596
          ],
          [
            2.146831,
            48.79069
          ],
          [
            2.146036,
            48.791391
          ],
          [
            2.142612,
            48.792427
          ],
          [
            2.141724,
            48.79271
          ],
          [
            2.140724,
            48.793014
          ],
          [
            2.140529,
            48.792538
          ],
          [
            2.140368,
            48.792086
          ],
          [
            2.139738,
            48.790388
          ],
          [
            2.139254,
            48.7891
          ],
          [
            2.139041,
            48.788508
          ],
          [
            2.138136,
            48.78905
          ],
          [
            2.137493,
            48.789417
          ],
          [
            2.135825,
            48.790407
          ],
          [
            2.135814,
            48.79014
          ],
          [
            2.135758,
            48.789939
          ],
          [
            2.135338,
            48.789591
          ],
          [
            2.13476,
            48.789073
          ],
          [
            2.134258,
            48.788669
          ],
          [
            2.133943,
            48.788516
          ],
          [
            2.133861,
            48.788224
          ],
          [
            2.133756,
            48.788144
          ],
          [
            2.133615,
            48.78779
          ],
          [
            2.133617,
            48.787731
          ],
          [
            2.13329,
            48.78777
          ],
          [
            2.13326,
            48.787654
          ],
          [
            2.132869,
            48.787796
          ],
          [
            2.132549,
            48.787405
          ],
          [
            2.131745,
            48.786529
          ],
          [
            2.131715,
            48.786482
          ],
          [
            2.130362,
            48.785038
          ],
          [
            2.128602,
            48.783122
          ],
          [
            2.127633,
            48.782388
          ],
          [
            2.125797,
            48.780911
          ],
          [
            2.124684,
            48.780059
          ],
          [
            2.124605,
            48.779902
          ],
          [
            2.123729,
            48.780004
          ],
          [
            2.123059,
            48.780068
          ],
          [
            2.12161,
            48.780224
          ],
          [
            2.120167,
            48.780356
          ],
          [
            2.119238,
            48.780453
          ],
          [
            2.118817,
            48.780486
          ],
          [
            2.118473,
            48.780535
          ],
          [
            2.118185,
            48.78054
          ],
          [
            2.117718,
            48.780616
          ],
          [
            2.115708,
            48.779245
          ],
          [
            2.115406,
            48.779166
          ],
          [
            2.11385,
            48.779867
          ],
          [
            2.113195,
            48.780171
          ],
          [
            2.112994,
            48.780226
          ],
          [
            2.112494,
            48.780451
          ],
          [
            2.112176,
            48.780523
          ],
          [
            2.111253,
            48.780461
          ],
          [
            2.110976,
            48.780422
          ],
          [
            2.109803,
            48.78016
          ],
          [
            2.107735,
            48.779656
          ],
          [
            2.106932,
            48.779428
          ],
          [
            2.106703,
            48.779472
          ],
          [
            2.106269,
            48.779933
          ],
          [
            2.105717,
            48.780691
          ]
        ]
      ]
    },
    "_score": 1
  }
]

[virgile-dev]

Which appears to work fine : https://umap.incubateur.anct.gouv.fr/fr/map/versailles_388

[davidbgk]

The contour exists for communes but not for departements? https://geo.api.gouv.fr/decoupage-administratif/departements

[virgile-dev]

I've sent sent a support request to data gouv to see if we could get the other administrative contours. I'll let you know what they answer.