CorrelAid / pystatis

MIT License
6 stars 1 forks source link

Handle missing units #102

Open pmayd opened 2 weeks ago

pmayd commented 2 weeks ago

It seems that it can happen that a unit is not known by the API. Normally, the unit for value colums is either returned as part of the label or, in the case of Zensus database, as an additional column value_unit. I found the follow case (52411-02-01-4)

Statistik_Code;Statistik_Label;Zeit_Code;Zeit_Label;Zeit;1_Merkmal_Code;1_Merkmal_Label;1_Auspraegung_Code;1_Auspraegung_Label;ISV006__Insolvenzverfahren_(Unternehmen)__MeasureUnitNotFound!;ISVNW1__Arbeitnehmer__Anzahl;FOR002__voraussichtliche_Forderungen_(Unternehmen)__Tsd._EUR

As you can see, there is one value column with an unknown unit: ISV006__Insolvenzverfahren_(Unternehmen)__MeasureUnitNotFound!. In this case it would be interesting to check the catalogue endpoint to retrieve the metadata for that variable and check the unit there. In any case we have to deal with this kind of missing information and give a warning to the user. One solution could be to replace this text by "Unknown"