johanmorganti / osm-datadog

Monitoring OpenStreetMap with Datadog
12 stars 1 forks source link

OSM Notes statistics #8

Open angoca opened 1 year ago

angoca commented 1 year ago

Have you thought about including some stats about notes? Something like:

In fact, there are many stats that can be extracted from notes. These are some examples (In Spanish) https://wiki.openstreetmap.org/wiki/ES:LatAm/Proyectos/Resoluci%C3%B3n_de_notas/Preparaci%C3%B3n_premios#11._Ejecutar_queries_para_el_an%C3%A1lisis_sobre_las_notas.

johanmorganti commented 1 year ago

That's indeed a good idea. Do you have any real-time stream source that we could leverage ?

Any other idea are welcome :)

angoca commented 1 year ago

The only way to get real-time stream for notes is with the API. Notes are not as active as OSM changes, so you can poll it each 15 minutes.

johanmorganti commented 1 year ago

Gotcha. The first version of my script was pooling the API, and some members told me to prevent doing this and use the replica instead. Since we don't have other solution available, and since that will be only every 15min, I believe we can go ahead with that way of getting fresh data.

At first glance, the issue I see is that I should use bbox with the API : https://wiki.openstreetmap.org/wiki/API_v0.6#Map_Notes_API And playing around, I got the error :

The maximum bbox size is 25, and your request was too large. Either request a smaller area, or use planet.osm

Not great, but that's a good start! Will continue later on

angoca commented 1 year ago

I think you should use: /api/0.6/notes/search

angoca commented 1 year ago

I am developing a tool to process notes that processes the planet notes file and the API. The scripts are at: https://github.com/OSMLatam/OSM-Notes-profile