search

NREL / reV

Renewable Energy Potential (reV) Model
https://nrel.github.io/reV/
BSD 3-Clause "New" or "Revised" License
107 stars 24 forks source link

reV-generation breaks with single hub-height wind resource datasets. #369

Closed WilliamsTravis closed 2 years ago

WilliamsTravis commented 2 years ago

Bug Description reV does not appear to be able to handle wind resource datasets with only one hub height.

Full Traceback

/home/twillia2/github/rex/rex/renewable_resource.py:820: ResourceWarning: Wind speed is only available at 120m, all variables will be extracted at 120m
  warnings.warn('Wind speed is only available at {h}m, '
concurrent.futures.process._RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/home/twillia2/miniconda3/envs/devruns/lib/python3.8/concurrent/futures/process.py", line 239, in _process_worker
    r = call_item.fn(*call_item.args, **call_item.kwargs)
  File "/home/twillia2/github/reV/reV/generation/generation.py", line 316, in run
    raise e
  File "/home/twillia2/github/reV/reV/generation/generation.py", line 310, in run
    out = cls.OPTIONS[tech].reV_run(points_control, res_file, site_df,
  File "/home/twillia2/github/reV/reV/SAM/generation.py", line 424, in reV_run
    resources = RevPySam.get_sam_res(res_file,
  File "/home/twillia2/github/reV/reV/SAM/SAM.py", line 553, in get_sam_res
    return SamResourceRetriever.get(*args, **kwargs)
  File "/home/twillia2/github/reV/reV/SAM/SAM.py", line 247, in get
    res = res_handler.preload_SAM(res_file, *args, **kwargs)
  File "/home/twillia2/github/rex/rex/renewable_resource.py", line 1144, in preload_SAM
    SAM_res = res._preload_SAM(sites, hub_heights,
  File "/home/twillia2/github/rex/rex/renewable_resource.py", line 1144, in preload_SAM
    SAM_res = res._preload_SAM(sites, hub_heights,
  File "/home/twillia2/github/rex/rex/renewable_resource.py", line 1060, in _preload_SAM
    SAM_res[var] = self[ds_name, time_slice, sites]
  File "/home/twillia2/github/rex/rex/renewable_resource.py", line 406, in __getitem__
    out = super().__getitem__(keys)
  File "/home/twillia2/github/rex/rex/resource.py", line 631, in __getitem__
    out = self._get_ds(ds, ds_slice)
  File "/home/twillia2/github/rex/rex/renewable_resource.py", line 930, in _get_ds
    out = self._get_ds_height(ds_name, ds_slice)
  File "/home/twillia2/github/rex/rex/renewable_resource.py", line 880, in _get_ds_height
    (h1, h2), extrapolate = self.get_nearest_h(h, heights)
  File "/home/twillia2/github/rex/rex/renewable_resource.py", line 522, in get_nearest_h
    h_min, h_max = np.sort(heights)[[0, -1]]
IndexError: index 0 is out of bounds for axis 0 with size 0
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/twillia2/miniconda3/envs/devruns/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/twillia2/miniconda3/envs/devruns/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/twillia2/github/reV/reV/generation/cli_gen.py", line 780, in <module>
    main(obj={})
  File "/home/twillia2/miniconda3/envs/devruns/lib/python3.8/site-packages/click/core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "/home/twillia2/miniconda3/envs/devruns/lib/python3.8/site-packages/click/core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "/home/twillia2/miniconda3/envs/devruns/lib/python3.8/site-packages/click/core.py", line 1659, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/twillia2/miniconda3/envs/devruns/lib/python3.8/site-packages/click/core.py", line 1659, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/twillia2/miniconda3/envs/devruns/lib/python3.8/site-packages/click/core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/twillia2/miniconda3/envs/devruns/lib/python3.8/site-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/home/twillia2/miniconda3/envs/devruns/lib/python3.8/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/twillia2/github/reV/reV/generation/cli_gen.py", line 431, in local
    Gen.reV_run(tech,
  File "/home/twillia2/github/reV/reV/generation/generation.py", line 563, in reV_run
    raise e
  File "/home/twillia2/github/reV/reV/generation/generation.py", line 558, in reV_run
    gen._parallel_run(max_workers=max_workers, pool_size=pool_size,
  File "/home/twillia2/github/reV/reV/generation/base.py", line 1168, in _parallel_run
    result = future.result(timeout=timeout)
  File "/home/twillia2/miniconda3/envs/devruns/lib/python3.8/concurrent/futures/_base.py", line 444, in result
    return self.__get_result()
  File "/home/twillia2/miniconda3/envs/devruns/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result
    raise self._exception
  File "/home/twillia2/miniconda3/envs/devruns/lib/python3.8/concurrent/futures/process.py", line 239, in _process_worker
    r = call_item.fn(*call_item.args, **call_item.kwargs)
  File "/home/twillia2/github/reV/reV/generation/generation.py", line 316, in run
    raise e
  File "/home/twillia2/github/reV/reV/generation/generation.py", line 310, in run
    out = cls.OPTIONS[tech].reV_run(points_control, res_file, site_df,
  File "/home/twillia2/github/reV/reV/SAM/generation.py", line 424, in reV_run
    resources = RevPySam.get_sam_res(res_file,
  File "/home/twillia2/github/reV/reV/SAM/SAM.py", line 553, in get_sam_res
    return SamResourceRetriever.get(*args, **kwargs)
  File "/home/twillia2/github/reV/reV/SAM/SAM.py", line 247, in get
    res = res_handler.preload_SAM(res_file, *args, **kwargs)
  File "/home/twillia2/github/rex/rex/renewable_resource.py", line 1144, in preload_SAM
    SAM_res = res._preload_SAM(sites, hub_heights,
  File "/home/twillia2/github/rex/rex/renewable_resource.py", line 1144, in preload_SAM
    SAM_res = res._preload_SAM(sites, hub_heights,
  File "/home/twillia2/github/rex/rex/renewable_resource.py", line 1060, in _preload_SAM
    SAM_res[var] = self[ds_name, time_slice, sites]
  File "/home/twillia2/github/rex/rex/renewable_resource.py", line 406, in __getitem__
    out = super().__getitem__(keys)
  File "/home/twillia2/github/rex/rex/resource.py", line 631, in __getitem__
    out = self._get_ds(ds, ds_slice)
  File "/home/twillia2/github/rex/rex/renewable_resource.py", line 930, in _get_ds
    out = self._get_ds_height(ds_name, ds_slice)
  File "/home/twillia2/github/rex/rex/renewable_resource.py", line 880, in _get_ds_height
    (h1, h2), extrapolate = self.get_nearest_h(h, heights)
  File "/home/twillia2/github/rex/rex/renewable_resource.py", line 522, in get_nearest_h
    h_min, h_max = np.sort(heights)[[0, -1]]
IndexError: index 0 is out of bounds for axis 0 with size 0
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "/home/twillia2/miniconda3/envs/devruns/lib/python3.8/concurrent/futures/process.py", line 102, in _python_exit
    thread_wakeup.wakeup()
  File "/home/twillia2/miniconda3/envs/devruns/lib/python3.8/concurrent/futures/process.py", line 90, in wakeup
    self._writer.send_bytes(b"")
  File "/home/twillia2/miniconda3/envs/devruns/lib/python3.8/multiprocessing/connection.py", line 183, in send_bytes
    self._check_closed()
  File "/home/twillia2/miniconda3/envs/devruns/lib/python3.8/multiprocessing/connection.py", line 136, in _check_closed
    raise OSError("handle is closed")
OSError: handle is closed

Sam Config (power curve abbreviated)

{
    "adjust:constant": 0,
    "capital_cost": 5005000,
    "fixed_operating_cost": 214225,
    "fixed_charge_rate": 0.049,
    "system_capacity": 5500.0,
    "variable_operating_cost": 0,
    "turb_generic_loss": 11.8,
    "wind_farm_wake_model": 0,
    "wind_farm_xCoordinates": [
        0
    ],
    "wind_farm_yCoordinates": [
        0
    ],
    "wind_resource_model_choice": 0,
    "wind_resource_shear": 0.14,
    "wind_resource_turbulence_coeff": 0.1,
    "wind_turbine_hub_ht": 120,
    "wind_turbine_powercurve_powerout": [0...],
    "wind_turbine_powercurve_windspeeds": [0...],
    "wind_turbine_rotor_diameter": 175
}

Resource File

/datasets/WIND/conus/v1.0.0/2012/wtk_conus_2012_120m.h5

Expected behavior reV-generation uses the singly available hub-height to generate generation profiles.

System (please complete the following information):

Charge code. Can request if not simple fix.

ppinchuk commented 2 years ago

@WilliamsTravis This is all excellent information, allowing me to understand your problem (and try to help fix it) quickly and easily.

Without running explicit tests myself (would probably want a charge code for that), I think the problem might be that your single height resource dataset is missing pressure data:

h5ls /datasets/WIND/conus/v1.0.0/2012/wtk_conus_2012_120m.h5 
coordinates              Dataset {2488136, 2}
meta                     Dataset {2488136}
temperature_120m         Dataset {105408, 2488136}
time_index               Dataset {105408}
winddirection_120m       Dataset {105408, 2488136}
windspeed_120m           Dataset {105408, 2488136}

reV (more correctly, SAM), needs pressure info to run generation. Note that the aggregated data have that included:

h5ls /datasets/WIND/conus/v1.0.0/wtk_conus_2012.h5 
coordinates              Dataset {2488136, 2}
inversemoninobukhovlength_2m Dataset {8784, 2488136}
meta                     Dataset {2488136}
precipitationrate_0m     Dataset {8784, 2488136}
pressure_0m              Dataset {8784, 2488136}
pressure_100m            Dataset {8784, 2488136}
pressure_200m            Dataset {8784, 2488136}
relativehumidity_2m      Dataset {8784, 2488136}
temperature_100m         Dataset {8784, 2488136}
temperature_10m          Dataset {8784, 2488136}
temperature_120m         Dataset {8784, 2488136}
temperature_140m         Dataset {8784, 2488136}
temperature_160m         Dataset {8784, 2488136}
temperature_200m         Dataset {8784, 2488136}
temperature_2m           Dataset {8784, 2488136}
temperature_40m          Dataset {8784, 2488136}
temperature_60m          Dataset {8784, 2488136}
temperature_80m          Dataset {8784, 2488136}
time_index               Dataset {8784}
winddirection_100m       Dataset {8784, 2488136}
winddirection_10m        Dataset {8784, 2488136}
winddirection_120m       Dataset {8784, 2488136}
winddirection_140m       Dataset {8784, 2488136}
winddirection_160m       Dataset {8784, 2488136}
winddirection_200m       Dataset {8784, 2488136}
winddirection_40m        Dataset {8784, 2488136}
winddirection_60m        Dataset {8784, 2488136}
winddirection_80m        Dataset {8784, 2488136}
windspeed_100m           Dataset {8784, 2488136}
windspeed_10m            Dataset {8784, 2488136}
windspeed_120m           Dataset {8784, 2488136}
windspeed_140m           Dataset {8784, 2488136}
windspeed_160m           Dataset {8784, 2488136}
windspeed_200m           Dataset {8784, 2488136}
windspeed_40m            Dataset {8784, 2488136}
windspeed_60m            Dataset {8784, 2488136}
windspeed_80m            Dataset {8784, 2488136}

I will close this issue for now on this basis. Please report back if you still have trouble running reV on single hub-height wind resource datasets that contain, at minimum, pressure, temperature, windspeed, and winddirection data.

grantbuster commented 2 years ago

This is great you guys, thank you both.

@WilliamsTravis, assuming you're running reV, try setting the resource filepath to /datasets/WIND/conus/v1.0.0/2012/wtk_conus_2012_*.h5. This should pick up all the 5min 2012 data including the hub heights with the pressure data. I think you can also set /datasets/WIND/conus/v1.0.0/{}/wtk_conus_{}_*.h5 which will iterate through the analysis years like normal (i havent tried this in a while but i think it works).

I'm going to copy this issue to rex where we should raise a much more obvious error if we dont have one of the key required variables.

WilliamsTravis commented 2 years ago

That totally worked. Bravo!

ppinchuk commented 2 years ago

Great point @grantbuster, I didn't even think of that.

~Just for my own curiosity, is using/datasets/WIND/conus/v1.0.0/2012/wtk_conus_2012_*.h5 any different than using /datasets/WIND/conus/v1.0.0/wtk_conus_2012.h5? I would think that neither of these are technically considered single hub-height wind resource data, but I could be wrong~

Nvm, different time scales

grantbuster commented 2 years ago

yeah the annual sub folders have 5min wtk data. The files are too big to have all hub heights in a single file so they get split up. The rex multi-file handlers are meant to deal with this by feeding them the unix style file*patterns