set_multiple_rc_from_isimip fails to write output

[lauzikaite]

Hi,

I've been trying to calculate the historical mean for the ISIMIP files according to the : Crop production risk based on ISIMIP and FAO data tutorial

The function set_multiple_rc_from_isimip fails with every netcdf file I tried when I select to save the output.

input_dir= "/data/bf1206/climada/data/agriculture/historical"
output_dir = "/data/bf1206/climada/data/agriculture/historical/Hist_mean"

print(os.listdir(input_dir))

filelist_haz, hazards_list = set_multiple_rc_from_isimip(input_dir = input_dir,
                                                         output_dir = output_dir,
                                                          isimip_run='ISIMIP2b',
                                                         yearrange_his = [1861,2005],
                                                         yearrange_mean = [1990,2005],
                                                         return_data = True, save = True)

['gepic_gfdl-esm2m_ewembi_historical_2005soc_co2_yield-mai-firr_global_annual_1861_2005.nc', 'Hist_mean', 'archive']

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
/tmp/ipykernel_6411/2690901969.py in <module>
      4 print(os.listdir(input_dir))
      5 
----> 6 filelist_haz, hazards_list = set_multiple_rc_from_isimip(input_dir = input_dir,
      7                                                          output_dir = output_dir,
      8                                                           isimip_run='ISIMIP2b',

/data/conda/bf1206/climada_env/lib/python3.8/site-packages/climada/hazard/relative_cropyield.py in set_multiple_rc_from_isimip(input_dir, output_dir, bbox, isimip_run, yearrange_his, yearrange_mean, return_data, save, combine_subcrops)
    486         else: output_list.append(None)
    487         if save:
--> 488             haz_his.select(reg_id=1).write_hdf5(str(Path(output_dir, 'Hazard', filename)))
    489 
    490         if isimip_run in ('ISIMIP2b', 'ISIMIP3b'):

/data/conda/bf1206/climada_env/lib/python3.8/site-packages/climada/hazard/base.py in select(self, event_names, date, orig, reg_id, reset_frequency)
    637         # filter centroids
    638         if reg_id is not None:
--> 639             sel_cen &= (self.centroids.region_id == reg_id)
    640             if not np.any(sel_cen):
    641                 LOGGER.info('No hazard centroids with region %s.', str(reg_id))

ValueError: operands could not be broadcast together with shapes (244800,) (0,) (244800,) 

running set_multiple_rc_from_isimip with save = False doesn't fail, but then I am not sure how the historical means shuld be saved into hdf5 files for the rest of the pipeline to work.

[sameberenz]

Hi @lauzikaite , could you please specify which branch / release / version of climada you are working on? Thanks

[lauzikaite]

Hi @sameberenz , I installed the lastest release 2.2.0 from the zip files directly, rather than the git repo.

[sameberenz]

Thanks! Unfortunately, I am not familiar with the error, let's try to nail it down:

Could you please test whether you have the same error in release 2.1.1 (https://github.com/CLIMADA-project/climada_python/releases/tag/v2.1.1) ?

Also: What kind / version of (ISIMIP) crop yield input data do you use?

[emanuel-schmid]

is there any update about this?

[sameberenz]

no update from my side. It should be noted that the set_multiple*-functions in the crop modules (exposure and hazard) are non-essential convenience wrappers that were written to apply the core methods of the modules to multiple files placed in a folder. They have not been maintained and tested to the same degree as the core functionalities. If they do not work, it is advisable to improve them or use them as inspiration for coding own wrapper functions tailored to the enviroment and use case at hand.