Closed zaneselvans closed 2 years ago
I listed additional placeholder values that need to be replaced with null
in https://github.com/catalyst-cooperative/pudl/pull/806#issue-509727837 and removed in the metadata, namely:
constraints.enum
for fields:
construction_type
fuel_unit
ENTITY_TYPES
.
In some of our categorical columns, at the end of the transform step we have only the enumerated values, plus the empty string. In the database, these columns are nullable, and so the empty string values should really be
pd.NA
. However, currently we have the empty string in some of our enumerations to avoid failing constraint checks. These should all be removed, and the corresponding columns should have the empty string replaced withpd.NA
before loading. Columns that this is happening in include:FERC 1
construction_type
EIA
natural_gas_delivery_contract_type_code
natural_gas_transport_code
purchase_type
EPA CEMS
co2_mass_measurement_code
nox_mass_measurement_code
nox_rate_measurement_code
so2_mass_measurement_code