search

proyectopetra / grx-PETRA

PETRA: Traffic Prediction and Monitoring
GNU General Public License v2.0
8 stars 1 forks source link

Convert Ministry of Works and Highways traffic maps to a more legible format #1

Closed fergunet closed 9 years ago

fergunet commented 9 years ago

Convert the Ministry's PDF to a more legible format (CSV, JSON...)

http://www.fomento.gob.es/MFOM/LANG_CASTELLANO/DIRECCIONES_GENERALES/CARRETERAS/TRAFICO_VELOCIDADES/DATOS_HISTORICOS/

stringparser commented 9 years ago

Pasados a texto plano, ahora a filtrarlos

fergunet commented 9 years ago

I've just uploaded the files to Datasets/dgt folder. We need to preprocess the files to facilitate its use in the other modules. I think we can skip the %CRECIMIENTO (as can be calculated from other cells) and focus only in number of vehicles; days (I don't know what does it means, but can be useful) and the station information (PK, province, etc.).

Files are: HPermanentes.csv HPrimarias.csv HSecundarias.csv HSemipermanentes.csv HPeajes.csv

I suggest to create only ONE csv with all information gathered to facilitate plotting, representation and classification.

For example a section of file HPermanentes.csv, passing from

;;;;;;;;;;;;EVOLUCIÓN HISTÓRICA DE UNA ESTACIÓN;;;;;
;;;;;;;Estación:;A-156-5 Calzada:;;;Total;;Carriles:;;1+1;Prov:;A
;;;;;;;Población:;VILLAJOYOSA (VARIANTE);;;;;Carretera:;;N-332;PK:;141,90
;;;;;;;;;;;Denominación antigua:;;;;N-332;;
;;;;IMD;;;;;;% CRECIMIENTO;;;;;FUNCIONAMIENTO;;
;;;;;;;%;;;;;;;No;;Ref. Año;Grado
Año;Total;Moto;Lige;Bus;Camión;Pesa;Pesa;Total;Moto;Lige;Bus;Camión;Pesa;Días;%;Ant;Comarac
2013;21572;5;20395;3;1169;1172;5.4;0.96;-14.29;1.96;0;-13.72;-13.69;000;0;;
2012;21366;6;20002;3;1355;1358;6.4;0.77;-98.77;2.98;-97.46;16.91;4.95;093;100;;
2011;21203;486;19423;118;1159;1294;6.1;-9.71;56.27;-10.06;-18.06;-17.86;-17.89;000;0;A-156-3;
etc.

to

Tipo;Año;Total;Moto;Lige;Bus;Camión;Pesa;Días;Estacion;Calzada;Carriles;Poblacion;Prov;Carretera;PK;
Permanentes;2013;21572;5;20395;3;1169;1172;5;000;A-156-5;Total;1+1;VILLAJOYOSA (VARIANTE);A;N-332;PK:141,90
Permanentes;2012;21366;6;20002;3;1355;1358;093;A-156-5;Total;1+1;VILLAJOYOSA (VARIANTE);A;N-332;PK:141,90
Permanentes;2011;21203;486;19423;118;1159;A-156-5;000;Total;1+1;VILLAJOYOSA (VARIANTE);A;N-332;PK:141,90
etc.

and the rest of the files, of course

Warning, some cells may have errors (I saw a value like "1222 63.3" instead "1222;63.33". So, manage spaces/tabs also as separators in the numeric values.

In principle @stringparser is assigned, but if anybody wants to do this go on :)

Then @unintendedbear @deantares can put it in WEKA, @afercab in one of the OLAP programs, @AnabelGRios and @JacintoCC in CartoDB, etc.

stringparser commented 9 years ago

ok, downloading them