openeventdatabase / datasources

scripts to populate OpenEventDatabase from public data sources
GNU Affero General Public License v3.0
20 stars 5 forks source link

fr.prix-carburants seems out of sync #7

Open PanierAvide opened 6 years ago

PanierAvide commented 6 years ago

Hi, It seems that there is a difference between what is available on https://www.prix-carburants.gouv.fr/ and what OpenEventDatabase shows through API. Let's take an exemple : station with ID 35850003.

API : http://api.openeventdatabase.org/event?what=fuel.price&bbox=-1.809955,48.207086,-1.769743,48.233821

{
  "count": 1,
  "features": [
    {
      "geometry": {
        "coordinates": [
          -1.79075395872,
          48.2211709682
        ],
        "type": "Point"
      },
      "properties": {
        "adresse": "Rue de Rennes",
        "carburants": [
          {
            "carburant": "Gazole",
            "carburant_id": "1",
            "maj": "2018-06-22 09:16:51",
            "prix": 1.406
          },
          {
            "carburant": "SP95",
            "carburant_id": "2",
            "maj": "2018-06-22 09:16:52",
            "prix": 1.505
          },
          {
            "carburant": "SP98",
            "carburant_id": "6",
            "maj": "2018-06-14 16:14:17",
            "prix": 1.544
          }
        ],
        "createdate": "2018-06-22T07:30:07.744543",
        "id": "53b4811d-d2b0-4340-8666-f1d3e02468f2",
        "id_pdv": "35850003",
        "label": "Gazole: 1.406€, SP95: 1.505€, SP98: 1.544€",
        "lastupdate": "2018-06-22T07:30:07.744543",
        "lat": 48.2211709682,
        "lon": -1.79075395872,
        "marque": "Carrefour Market",
        "nom": "Carrefour Market",
        "ouverture:fr": "24/7",
        "ouverture_debut": "01:00",
        "ouverture_fin": "01:00",
        "ouverture_sauf": "",
        "pop": "R",
        "postcode": "35850",
        "prev_event": "http://api.openeventdatabase.org/event/5fc4ca2e-5cf6-45ec-8e25-1cf415b59456",
        "services": [
          "Vente de gaz domestique"
        ],
        "source": "www.prix-carburants.gouv.fr",
        "start": "2018-06-22 09:16:52",
        "stop": "2018-07-20T07:30:02.317196",
        "type": "observed",
        "ville": "GéVEZé",
        "what": "fuel.price",
        "where_name": "Carrefour Market, Rue de Rennes 35850 GéVEZé"
      },
      "type": "Feature"
    }
  ],
  "type": "FeatureCollection"
}

Official website capture d ecran_2018-07-09_13-34-06

And of course ground truth is what is shown on prix-carburants.gouv.fr. Is there any explanation for this behaviour ? Am I missing something from API call, or is there a broken data pipeline ? Regards.

PanierAvide commented 6 years ago

Not sure if linked, but no result at all is available for fuel prices now.

DuchkPy commented 2 years ago

@PanierAvide the gouvernement website as been updated several times since this code was created. Each time it breaks something.

I made a PR #11 but even since, gouverenement website as changes and breaks stuff again.

Also, I have a fork that is up to date

PanierAvide commented 2 years ago

Well now I use open data XML which is published on their website (which is quite stable but a pain to read), and put a GeoJSON processed version here (updated daily) : https://files.pavie.info/depot/remote/carburants_gouv.geojson

There is also a processed version which uses OpenStreetMap data, in particular for geolocation of stations which are more precise when available : https://files.pavie.info/depot/remote/carburants.geojson

These might be a simpler solution for integration in OEDB ?