Fix Nightly Build Failure 2023-12-28

[zaneselvans]

Outputs: gs://nightly-build-outputs.catalyst.coop/2023-12-28-0627-ef8ab93-dev

New failure in the integration tests when run against all years of data.

I note that there's a discrepancy between the error message and the actual check:

>       assert ratio_correct > 0.95, "Percent of correctly matched FERC records below 85%."
E       AssertionError: Percent of correctly matched FERC records below 85%.
E       assert 0.8571428571428571 > 0.95

Which were we intending to check?

__________________________ test_classify_plants_ferc1 __________________________
[gw0] linux -- Python 3.11.7 /home/mambauser/env/bin/python3.11

mock_ferc1_plants_df =      index       base_plant_name plant_type  report_year construction_type  capacity_mw  construction_year  utility_id...            NaN                    NaN                NaN                  NaN               0

[546 rows x 15 columns]

    def test_classify_plants_ferc1(mock_ferc1_plants_df):
        """Test the FERC inter-year plant linking model."""

        @graph
        def _link_ids(df: pd.DataFrame):
            feature_matrix = ferc_dataframe_embedder(df)
            label_df = link_ids_cross_year(df, feature_matrix)
            return label_df

        mock_ferc1_plants_df["plant_id_ferc1"] = (
            _link_ids.to_job()
            .execute_in_process(input_values={"df": mock_ferc1_plants_df})
            .output_value()["record_label"]
        )

        # Compute percent of records assigned correctly
        correctly_matched = (
            mock_ferc1_plants_df.groupby("base_plant_name")["plant_id_ferc1"]
            .apply(lambda plant_ids: plant_ids.value_counts().iloc[0])
            .sum()
        )
        ratio_correct = correctly_matched / len(mock_ferc1_plants_df)
        logger.info(f"Percent correctly matched: {ratio_correct*100:.2f}%")
>       assert ratio_correct > 0.95, "Percent of correctly matched FERC records below 85%."
E       AssertionError: Percent of correctly matched FERC records below 85%.
E       assert 0.8571428571428571 > 0.95

Also @zschira numba is putting out like 16,000 lines of debugging logs when running this test, which I don't think we want to see. How can we silence it?

[zaneselvans]

My guess is that @zschira caught this failure and intended update both the error message and the numerical threshold, but only changed one of them. So maybe we just need to update the numerical threshold.

[zaneselvans]

Went ahead and merged this fix in so we get another go at the nightly builds tonight...