Reconcile ownership_types and entity_types in EIA 860/861

[zaneselvans]

The EIA 860 has a column called entity_type and the EIA 861 has a column called ownership_type. They take on very similar but not identical values, and seem to refer to the same utility attribute. We should figure out if they can be reconciled with each other as actually being the same thing, both for simplicity, and so one can merge based on them. This is more a research question than a coding issue -- maybe a call directly to EIA would be helpful?

# Historically we have not translated these codes to their long-form types.
# However, I just made this change locally...
entity_types_eia860 = {
    "C": "Cooperative",
    "F": "Federal",
    "I": "Investor Owned",
    "M": "Municipal",
    "P": "Political Subdivision",
    "S": "State",
    # These are not in EIA 861:
    "Q": "Independent Power Producer",
    "IND": "Industrial",
    "COM": "Commercial",
}

ownership_types_eia861 = {
    "Cooperative",
    "Federal",
    "Investor Owned",
    "Municipal",
    "Political Subdivision",
    "State",
    # These are not in EIA 860:
    "Retail Power Marketer",
    "Unregulated",
    "Behind the Meter",
    "Community Choice Aggregator",
    "Wholesale Power Marketer",
}

[cmgosnell]

These utility types are different because of who is supposed to be reporting to each of the EIA forms. And IPP is not a utility in the 861 sense. They are producing power from a plant; same with commercial and industrial. Likewise on the 860 types; these entities do not generate power so they are not in 860 but they do buy and sell power and thus are required to report in 861.

[zaneselvans]

Are you saying that the entity/ownership types are coming from the same list of labels, and that the only reason some of the labels are missing from one of the data sources is that that type of entity is not required to report that kind of data? I.e. that IPPs, IND, and COM produce electricity (and so have to report on EIA 860) but do not behave like utilities (and thus do NOT report on EIA 861), while the PPAs, Power Marketers, Unregulated, and "Behind the Meter" entities don't generate power, but do act like utilities, so they have to report on EIA 861, but not on EIA 860?

[cmgosnell]

Yes. This seems like expected behavior.

[zaneselvans]

I think I agree on the IPPs, COM, and IND types being power producers that aren't utilities, and on the CCAs and Power Marketers being utility-like things that don't produce power. I feel weird about Unregulated and "Behind the Meter" though. I've definitely seen utilities in the EIA 860 or 923 that are categorized as Regulated vs Unregulated. And what do they mean by "Behind the Meter" -- why is that even a thing in the utility reporting side of things?

[cmgosnell]

BTM is a new addition to 860. And idk about Unregulated. That seems like an odd thing to be in this column all together but I would have to look into the documentation to really understand. Regardless, this whole issue.. doesn't seem like an issue to me.

[zaneselvans]

So you think we should treat these as the same column, with the same name, with some values that show up in both, and some values which are limited only to one or the other of the forms?

[cmgosnell]

exactly.

[zaneselvans]

Okay great! I hope that's correct. It would be really nice to have them connected.

[zaneselvans]

I will go with what feels like the more general column name of entity_type