search

Ninja-Squad / globe42

repository dedicated to the development of Globe 42 collective application to help them to manage their members
12 stars 3 forks source link

add neighborhood on Person #81

Closed acrepet closed 4 years ago

acrepet commented 7 years ago

We have to add a new information in the Person form : his/her district / neighborhood (Carnot, Beaubrun, Chavanelle, ... for Saint-Etienne for example) and just for some cities : Saint-Etienne, Saint-Chamond, L'Horme, Le Chambon Feugerolles, Firminiy, Unieux.

As @jnizet did for #67 we would use a free dataset to populate our database with cities... For Saint-Etienne, the dataset is freely available, in CSV format (from data.gouv.fr) but I found some problems about the quality of these data : for example some district names are wrong in the dataset: Chataucreux -> ChatEaucreux ;-( We have these free datasets for the other cities : Saint-Chamond, Le Chambon Feugerolles, Firminy, Unieux except L'Horme.

Another hassle : we have to know if a district is among the "QVP districts" (quartiers prioritaires réglementaires) or the "QVA districts" (quartiers veille active) and there is not these information in the free dataset.

Extract of a legal text : **Les périmètres des 15 QPV reconnus par l’État sont fixés par le décret n° 2014-1750 du 30 décembre 2014 pour la métropole de Saint-Etienne. Ils peuvent bénéficier de la mobilisation préalable du droit commun, des crédits politique de la ville de tous les financeurs du contrat de ville et de tous les avantages automatiquement associés à ce statut. Ces quartiers stéphanois sont QVP : La Cotonne/Montferré Crêt de Roc/Soleil Montreynaud Quartiers Sud-Est Tarentaize/Beaubrun Terrenoire

Les 15 quartiers « en veille active » (QVA) également reconnus par l’État ne peuvent plus bénéficier des crédits spécifiques de l’État au titre de la politique de la ville mais sont identifiés comme rencontrant des difficultés. A ce titre, ils pourront bénéficier d’un accompagnement technique et/ou d’un soutien financier au titre du droit commun pour l’État et des crédits politique de la ville des autres partenaires signataires du contrat de ville. Ces quartiers stéphanois sont QVA : Chavanelle-Saint-Roch La Dame blanche Solaure Soleil Tardy "

My proposal is to populate manually a district table with something like that for the columns :

acrepet commented 7 years ago

Yesterday at the demo with Malika, from Globe, she said us that it could be nice to know if an address matches to a district QVP or QVA. Malika has to give us the URL she uses to know that. Perhaps we could integrate an open API

acrepet commented 7 years ago

this is the website used by Malika :

Here is a form to know if an address matches with a QVP (I have not yet the information for QVA) : https://sig.ville.gouv.fr/recherche-adresses-qp-polville the request is a POST with few parameters :
insee_com (hidden, fixed -> 69091) code_postal nom_commune num_adresse id_voie (hidden) nom_voie

jnizet commented 7 years ago

I have tried playing a bit with this shitty form, and I really don't think we can automate anything, except adding a link to open the page in a new tab. It requires not only the postal code, but also the city, with exactly the same case as they expect, and same for the other fields. There doesn't seem to be a way to prepopulate the form, and it only returns HTML, which would be horrible to parse safely.

cexbrayat commented 7 years ago

I did not really follow, but can't we do the same as we did for the address, build our own API based on open data? Something like https://www.data.gouv.fr/fr/datasets/quartiers-prioritaires-politique-de-la-ville/ ?

Update: can't find the data for Loire...

jnizet commented 7 years ago

That would be ideal. I didn't know that existed, and have no idea if it's the same dataset as the one used by the link posted by Agnès. But even if we do that, that's not an easy task: we would have to geolocate the address, then test if the geolocation is inside one of the polygons inside the file. It's certainly doable using google maps APIs, but it won't be a piece of cake. It can be fun to try doing it, but maybe we should start by something simpler.

cexbrayat commented 7 years ago

I can't find the data for Loire, Rhone-Alpes or France anyway, it seems that only some cities and departments have uploaded theirs...

acrepet commented 7 years ago

Yes, I have found these data from data.gouv but you don't have the same for St-Etienne and its suburbs. You're right @cexbrayat : only some cities and departments have uploaded their data!

acrepet commented 7 years ago

The magic of Twitter : I sent a tweet to data.gouv.fr and few hours after I had the answer :https://twitter.com/agnes_crepet/status/896380116640681984?s=09

They (data.gouv.fr) have an API in beta :

https://etalabapi-quartiers-prioritaires-hornzrkvrj.now.sh/quartiers-prioritaires?lat=49.116425128219795&lon=6.223948001861572

Their recommendations (from @jdesboeufs) : "Il faut chaîner avec un appel à l’API de géocodage https://t.co/MKIHCikLBT " "Il suffit de donner lat et lon, et l’API retourner la liste des QP concernés" "On va rajouter un paramètre permettant de définir une tolérance, en mètres, car l’adresse n’est pas toujours correctement positionnée." "Si vous avez besoin de plus de fonctions, faites-le moi savoir, nous pouvoir l’adapter rapidement."

acrepet commented 7 years ago

I've tested these APIs. I've chosen 2 addresses:

I've chained the two APIs, firstly the API of adresses search (https://api-adresse.data.gouv.fr/search) and then the beta API of QV (https://etalabapi-quartiers-prioritaires-hornzrkvrj.now.sh/quartiers-prioritaires):

For the first address (no QV) :
Calling this URL : https://api-adresse.data.gouv.fr/search/?q=10%20avenue%20de%20la%20liberation%2042000%20SAINT-ETIENNE

return : 

` {
    "version": "draft",
    "features": [{

            "geometry": {"type": "Point", "coordinates": [4.39203, 45.437121]},

            "type": "Feature",

            "properties": {"id": "ADRNIVX_0000000279715616", "importance": 0.3333, "name": "10 Avenue de la Lib\u00e9ration", "postcode": "42000", "housenumber": "10", "citycode": "42218", "x": 808829.6, "label": "10 Avenue de la Lib\u00e9ration 42000 Saint-\u00c9tienne", "y": 6482924.6, "street": "Avenue de la Lib\u00e9ration", "context": "42, Loire, Auvergne-Rh\u00f4ne-Alpes (Rh\u00f4ne-Alpes)", "type": "housenumber", "city": "Saint-\u00c9tienne", "score": 0.939390909090909}

       }],

    "licence": "ODbL 1.0",

    "attribution": "BAN",

    "query": "10 avenue de la liberation 42000 Saint-etienne",

    "type": "FeatureCollection",

    "limit": 5

    }`

The coordinates are the information that interests us : [4.39203, 45.437121] First parameter of these coordinates is the lon (longitude) and the second is the lat (latitude) : we can use them to call the second API which requires these parameters : lon & lat : https://etalabapi-quartiers-prioritaires-hornzrkvrj.now.sh/quartiers-prioritaires?lat=45.437121&lon=4.39203

This request returns no data: [] which means that this adresse doesn't match a QV

For the second address :
Calling this URL : https://api-adresse.data.gouv.fr/search/?q=30%20boulevard%20antonio%20vivaldi%2042000%20SAINT-ETIENNE

return these coordinates": [4.402135, 45.470242]}

We can use them to call the second API : https://etalabapi-quartiers-prioritaires-hornzrkvrj.now.sh/quartiers-prioritaires?lat=45.470242&lon=4.402135

The result is : [{"nom":"Montreynaud","code":"QP042013"}]

which means that this second address matches a QV called Montreynaud with the QV code (not mandatory for Gobe42) : QP042013

what do you think about that @jnizet & @cexbrayat ?

acrepet commented 7 years ago

Remember that firstly we had to know if a district is among the "QV" (quartiers prioritaires) or the "QVA" (quartiers veille active). But I had a last information from Malika this week : she never searches the QVA information from the website she gave us. She usually fill this QVA information manually. This is a good thing, because we don't have this information from data.gouv.fr ;-)

My first proposal was to have a district table with something like that for the columns :

But I think having a dedicated table of district is not necessary now ! We can just add a District information just below the address of the "Contact" section of a person (between for example Ville and Email), with two checkbox (or flags?) -> one for QVA and one for QVP For the QVA information, the user has to fill this information manually (check the checkbox or not), but for the QVP we can fill this information after having call of both the APIs as stated above in this issue (and the name of the QVP district in addition if we find by the APIs a result).