Add dimensions and tabular calculations to XBRL Calculation Forests

[zaneselvans]

With #2721 merged, the additional dimensions utility_type, plant_status, and plant_function are available in the newly compiled tabular calculation components table and in the exploded dataframes.

In order to validate and debug the calculations that involve these dimensions, and use them in analysis of the exploded data, they need to be integrated into the calculation forests, and the leaf-based filtering of the exploded data.

Rather than identifying a calculation component by (table_name, xbrl_factoid) we'll now need to use a tuple that also includes all of the other dimension columns. In many cases these will include null values, since not all dimensions are involved in all calculations. This larger tuple will also be used to filter the final exploded data table.

- [x] Use calc components table to construct calculation tree.
- [x] Use additional dimensions to filter the exploded dataframe.

[zaneselvans]

Questions

• In the exploded calculations there's only one set of columns for the additional dimensions. Do those values always apply to both the calculation component and the higher level fact that they're being aggregated into? That seems like it can't be true, since sometimes we'll have a calculated value that potentially sums up different kinds of values from different dimensions in different tables depending on e.g. utility_type.
• At first glance it seems like the exploded calculations should replace the exploded metadata. They seem to contain all the same information. But the exploded metadata contains all the directly reported (uncalculated) values too, while the calculation components only contain calculated values. Neither of them seems to contain all the information that's required to build the tree. Is there a reason for this information to be kept separate? Why not just have a single table output here with all the required information? Or if there are other applications, separate tables designed for use in those applications?
• Does the balance column still have meaning with all of the mucking around we've done in the calculations?
• What does the subdimension column mean? What values are allowed in it?
• There are a handful of electric values found in the plant_function column, but I don't think that's a valid value, based on the enumerated values in pudl.transform.ferc1.params.PLANT_FUNCTION_CATEGORIES
• What became of the source_tables lists? Which column were they exploded into?
• utility_type and plant_status show up in the balance sheet assets explosion, but not in the liabilities explosion. Does that matter? They seem kind of analogous to each other.

[cmgosnell]

• In the exploded calculations there's only one set of columns for the additional dimensions. Do those values always apply to both the calculation component and the higher level fact that they're being aggregated into? That seems like it can't be true, since sometimes we'll have a calculated value that potentially sums up different kinds of values from different dimensions in different tables depending on e.g. utility_type.
  • the dimensions apply to the calculation components not the higher level fact.
• At first glance it seems like the exploded calculations should replace the exploded metadata. They seem to contain all the same information. But the exploded metadata contains all the directly reported (uncalculated) values too, while the calculation components only contain calculated values. Neither of them seems to contain all the information that's required to build the tree. Is there a reason for this information to be kept separate? Why not just have a single table output here with all the required information? Or if there are other applications, separate tables designed for use in those applications?
  • I've been wondering this myself: whether we should rip out the factoid-level metadata in lieu of the calc components. But i don't think we should. namely because there are some things in there that are really much more factoid-level: like ferc_account for instance. Right now both tables (the factoid-level and calc-component-level tables) are very overlapping. I did this semi-intentionally bc I wasn't sure where we would need which piece of info. I think we should sort that out through the implementation of this issue.
• Does the balance column still have meaning with all of the mucking around we've done in the calculations?
  • NO! And knowing what we know (that a fact can be used in multiple different calculations with different signs....) i'm not sure its relevant as a column at all.
• What does the subdimension column mean? What values are allowed in it?
  • i thought i had fully removed all of the subdimension's but we had many prs happening so this probably got merged in. this shouldn't be there we

[zaneselvans]

I'm struggling with the new calculation_components_xbrl_ferc1 association table. The MetadataExploder.calculations() method needed to be updated to appropriately manage calculations that refer to components from outside of the explosion (turning them into parent-only calculation components) but either the inputs don't look like I think they do, or I'm doing something wrong in the new method, or my expectations about what should be true about the dataframe that's coming out of the method are wrong. See #2763