os-data / eu-structural-funds

The EU structural funds datasets on regional and national level (in progress).
26 stars 15 forks source link

Amounts incorrectly transposed #357

Closed michaeljokf closed 7 years ago

michaeljokf commented 7 years ago

we found out why there are some amounts that are so ridiculously high (e.g. Italy, Czech Republic, Germany...). We think it is still due to problems with files that have different delimiters. If this is the case then it should hopefully be fixable and give us the desired "reasonable" amounts.

More specific: e.g. Italy 2007-2013 the case of "Logimat 2" (project name) the total amount in the original CSV is 562065867,78 (so 562 million) - the amount displayed on the website and also in OpenSpending is 56.21 Billions. This leads us to believe that the two decimal characters were included in the amount on OS. Additionally, a lot of the other numbers in the total_amount column (OC_TOT_PAGAMENTI_RENDICONTAB_UE) do not have any decimal characters - and no delimiter. So this might be where the error comes from.

Other examples:

I will add more info on other countries and examples now - but for both cases above this holds not only for the one project but for multiple - just chose to use that example for illustration.

annafralberts commented 7 years ago

Checked comments the website, and the concat datasets etc. and the numbers are still not transposed correctly. @akariv can you try another fix and rerun them again?