fix_tidy_iea_df_balances() not working with Spain data

[earamendia]

Hi there,

I am getting an issue when trying to fix energy balances for Spain (2019 IEA data). It looks like the error in IEA balances is too important to be fixed. I don't know what would be the approach then?

Here's my code:

Tidy_ES <- IEA_data %>% 
  filter(COUNTRY == "Spain") %>%
  rename_iea_df_cols() %>% 
  use_iso_countries() %>% 
  remove_agg_memo_flows() %>% 
  augment_iea_df() %>% 
  tidy_iea_df()

Tidy_ES %>% 
  fix_tidy_iea_df_balances() %>% 
  calc_tidy_iea_df_balances() %>% 
  glimpse()

Error: Maximum energy balance error is 69186.4314. Maximum fixable error is 5, so we're stopping in fix_tidy_iea_df_balances

Cheers, Emmanuel

[MatthewHeun]

Try this in the PFU-Database repository:

1. Source the init.R file.
2. Run the drake workflow (choose "Run a drake workflow" from the "Addins" menu).
3. Type the following at the console:

   Tidy_ES <- readd_by_country(target = "Specified", country = "ESP") %>%
   fix_tidy_iea_df_balances() %>%  
   calc_tidy_iea_df_balances() %>% 
   glimpse()

   Everything works fine for me. Does it work for you? Are you doing something different?

If you want to try using only IEATools, you can do this (obviously fix the path for your computer):

AllIEAData <- IEATools::load_tidy_iea_df(file.path("~", "Dropbox", 
                                                   "Fellowship 1960-2015 PFU database", 
                                                   "IEA extended energy balance data", 
                                                   "IEA 2019 energy balance data", 
                                                   "IEA Extended Energy Balances 2019.csv"))

IEAData_ESP <- AllIEAData %>% 
  # Note eliminating 2018, which is an incomplete year.
  dplyr::filter(Country == "ESP", Year < 2017)

balanced_before_fix <- IEAData_ESP %>% 
  dplyr::group_by(Country, Method, Energy.type, Last.stage, Year, Product) %>%  
  IEATools::calc_tidy_iea_df_balances() %>% 
  IEATools::tidy_iea_df_balanced()
balanced_before_fix

should_be_balanced <- IEAData_ESP %>% 
  dplyr::group_by(Country, Method, Energy.type, Last.stage, Year, Product) %>%  
  IEATools::fix_tidy_iea_df_balances()

balanced_after_fix <- should_be_balanced %>% 
  dplyr::group_by(Country, Method, Energy.type, Last.stage, Year, Product) %>%  
  IEATools::calc_tidy_iea_df_balances() %>% 
  IEATools::tidy_iea_df_balanced()
balanced_after_fix

Let me know if this works for you.

[earamendia]

Absolutely, working now, thanks!

Actually the issue came from the latest year (2018) in the IEA data, so actually filtering like this;

IEAData_ESP <- AllIEAData %>% 
  dplyr::filter(Country == "ESP", Year < 2018) # not 2017

is enough.

[MatthewHeun]

Good catch on 2017, 2018. I'm closing for now.