Traffic APIs study

[fergunet]

I suggest @M42 and @fdavidcl but you can self-assign to other issues if you want: Some traffic API examples:

WAZE https://www.waze.com/es/about/dev Microsoft https://msdn.microsoft.com/en-us/library/hh441725.aspx Nokia HERE https://developer.here.com/rest-apis/documentation/traffic/topics/introduction.html EuropaPress http://www.europapress.es/trafico

Do you know others?

[JJ]

El escrapeo de europapress/trafico lo tenéis en el curso de Perl, pero @fdavidcl y @m42 ya estuvieron en la charla, así que supongo que se acordarán.

[fergunet]

Cool. By the way, we are going to switch issues to English to facilitate the judges work :)

We should translate our previous comments, also :)

[fdavidcl]

Alright. I've already investigated these APIs a bit:

• Waze's API seems to be exclusive for iOS/Android applications running on a device with Waze, although Waze is open source and we could try to simulate its requests.
• HERE's API needs a paid plan to obtain traffic data (https://developer.here.com/get-started#/10134035)
• Microsoft's API works, it needs a free application key. I've written an R function behaving as a wrapper to obtain data, I still have to organize it into a data.frame instead of a list.
• Europa Press works via POST requests, which I'm trying to imitate from R but I still have to work out how to correctly pass the parameters. There's a function that obtains a data.frame out of the HTML code from the response.

Do you want me to upload the code somewhere in the repo?

[fdavidcl]

Apparently a request with parameters to Europa Press is of the type:

curl --data-urlencode "__VIEWSTATE=%2FwEPDwUKMjEyNjQ5ODQ4MGRkU2VbKJPokWLsSlLEjOkZUVJoWVM%3D&__VIEWSTATEGENERATOR=C0A6878A&ctl00%24ContenidoCentral%24ddlCCAA=&ctl00%24ContenidoCentral%24txtCarretera=&ctl00%24ContenidoCentral%24ddlNiveles=&ctl00%24ContenidoCentral%24ddlProvincia=&ctl00%24ContenidoCentral%24ddlIncidencias=Todas&ctl00%24ContenidoCentral%24btnBuscar=Buscar" -v http://www.europapress.es/trafico

But this doesn't return any data, nor a redirection. Any incorrect POST request (not providing the __VIEWSTATE and __VIEWSTATEGENERATOR) will return a HTML with general data from everywhere in Spain.

[JJ]

You're probably better off just scraping.

[stringparser]

curl http://www.dgt.es/incidencias.xml

ver abajo algo que parece una api https://datospublicos.jottit.com/tr%C3%A1fico_y_transporte

[mroman42]

I have just added a script scraping the data in http://infocar.dgt.es/etraffic/. 78f9080b But it would be great to find a simpler way to get the data.

[stringparser]

I think I found something

curl http://www.dgt.es/incidencias.xml\?fecha\=20090101 > incidencias.xml
cat incidencias.xml | grep '2009'

with a date earlier than 2009 I can't found anything :)

[stringparser]

hahaha, I wrote in spanish :), fixed anyhow lets parse that xml thing

[fdavidcl]

No problem @stringparser, I'm adapting the R function to parse that XML instead and will upload it as soon as it works. Thanks for the links! :grin:

[stringparser]

great! :) I'll do it too just because we were all the morning parsing that pdfs ahahhaha

[JJ]

You are a bunch of f*cking geniuses!

[stringparser]

By the way the ?fecha= didn't work yesterday. Is weird, there is some 2009 dates there when you do that but it isn't really the history file from 2009.