search

NREL / reV

Renewable Energy Potential (reV) Model
https://nrel.github.io/reV/
BSD 3-Clause "New" or "Revised" License
107 stars 24 forks source link

Error for `reV-gen` / `pvwattsv7`: Failed to process irradiation on surface (code: -102) #383

Closed netr0m closed 1 year ago

netr0m commented 2 years ago

Bug Description

This may not necessarily be a bug with the source code, but we're having issues identifying the cause/problem

We've created our own resource file (data gathered from the "ERA5" dataset), following the structure of the brazil_solar.h5 example dataset. As such, our dataset contains:

When running reV gen (see example code below), an error occurs stating that an error occurred while processing irradiation on surface (code -102), but we're unable to identify what the exact issue is (Failed to process irradiation on surface (code: -102) [y:2016 m:1 d:1 h:0 minute:0].).

The H5 resource file can be shared on request. Hopefully you might be able to assist us in identifying the specific dataset in responsible for the error.

Full Traceback Full traceback of any errors received, if applicable

PySAM raised an error while executing: "pvwattsv7" for site 0
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/reV/SAM/SAM.py", line 775, in execute
    self.pysam.execute()
Exception: pvwattsv7 execution error.
        exec fail(pvwattsv7): Failed to process irradiation on surface (code: -102) [y:2016 m:1 d:1 h:0 minute:0].

Worker failed for PC: PointsControl with 4 sites from gid 0 through 3
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/reV/SAM/SAM.py", line 775, in execute
    self.pysam.execute()
Exception: pvwattsv7 execution error.
        exec fail(pvwattsv7): Failed to process irradiation on surface (code: -102) [y:2016 m:1 d:1 h:0 minute:0].

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/reV/generation/generation.py", line 310, in run
    out = cls.OPTIONS[tech].reV_run(points_control, res_file, site_df,
  File "/usr/local/lib/python3.9/dist-packages/reV/SAM/generation.py", line 460, in reV_run
    sim.run_gen_and_econ()
  File "/usr/local/lib/python3.9/dist-packages/reV/SAM/generation.py", line 357, in run_gen_and_econ
    self.run()
  File "/usr/local/lib/python3.9/dist-packages/reV/SAM/generation.py", line 382, in run
    self.execute()
  File "/usr/local/lib/python3.9/dist-packages/reV/SAM/SAM.py", line 782, in execute
    raise SAMExecutionError(msg) from e
reV.utilities.exceptions.SAMExecutionError: PySAM raised an error while executing: "pvwattsv7" for site 0
reV generation failed!
concurrent.futures.process._RemoteTraceback:
"""
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/reV/SAM/SAM.py", line 775, in execute
    self.pysam.execute()
Exception: pvwattsv7 execution error.
        exec fail(pvwattsv7): Failed to process irradiation on surface (code: -102) [y:2016 m:1 d:1 h:0 minute:0].

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/lib/python3.9/concurrent/futures/process.py", line 243, in _process_worker
    r = call_item.fn(*call_item.args, **call_item.kwargs)
  File "/usr/local/lib/python3.9/dist-packages/reV/generation/generation.py", line 316, in run
    raise e
  File "/usr/local/lib/python3.9/dist-packages/reV/generation/generation.py", line 310, in run
    out = cls.OPTIONS[tech].reV_run(points_control, res_file, site_df,
  File "/usr/local/lib/python3.9/dist-packages/reV/SAM/generation.py", line 460, in reV_run
    sim.run_gen_and_econ()
  File "/usr/local/lib/python3.9/dist-packages/reV/SAM/generation.py", line 357, in run_gen_and_econ
    self.run()
  File "/usr/local/lib/python3.9/dist-packages/reV/SAM/generation.py", line 382, in run
    self.execute()
  File "/usr/local/lib/python3.9/dist-packages/reV/SAM/SAM.py", line 782, in execute
    raise SAMExecutionError(msg) from e
reV.utilities.exceptions.SAMExecutionError: PySAM raised an error while executing: "pvwattsv7" for site 0
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/reV/generation/generation.py", line 558, in reV_run
    gen._parallel_run(max_workers=max_workers, pool_size=pool_size,
  File "/usr/local/lib/python3.9/dist-packages/reV/generation/base.py", line 1168, in _parallel_run
    result = future.result(timeout=timeout)
  File "/usr/lib/python3.9/concurrent/futures/_base.py", line 440, in result
    return self.__get_result()
  File "/usr/lib/python3.9/concurrent/futures/_base.py", line 389, in __get_result
    raise self._exception
reV.utilities.exceptions.SAMExecutionError: PySAM raised an error while executing: "pvwattsv7" for site 0
concurrent.futures.process._RemoteTraceback:
"""
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/reV/SAM/SAM.py", line 775, in execute
    self.pysam.execute()
Exception: pvwattsv7 execution error.
        exec fail(pvwattsv7): Failed to process irradiation on surface (code: -102) [y:2016 m:1 d:1 h:0 minute:0].

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/lib/python3.9/concurrent/futures/process.py", line 243, in _process_worker
    r = call_item.fn(*call_item.args, **call_item.kwargs)
  File "/usr/local/lib/python3.9/dist-packages/reV/generation/generation.py", line 316, in run
    raise e
  File "/usr/local/lib/python3.9/dist-packages/reV/generation/generation.py", line 310, in run
    out = cls.OPTIONS[tech].reV_run(points_control, res_file, site_df,
  File "/usr/local/lib/python3.9/dist-packages/reV/SAM/generation.py", line 460, in reV_run
    sim.run_gen_and_econ()
  File "/usr/local/lib/python3.9/dist-packages/reV/SAM/generation.py", line 357, in run_gen_and_econ
    self.run()
  File "/usr/local/lib/python3.9/dist-packages/reV/SAM/generation.py", line 382, in run
    self.execute()
  File "/usr/local/lib/python3.9/dist-packages/reV/SAM/SAM.py", line 782, in execute
    raise SAMExecutionError(msg) from e
reV.utilities.exceptions.SAMExecutionError: PySAM raised an error while executing: "pvwattsv7" for site 0
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/app/test.py", line 36, in <module>
    data = run()
  File "/app/test.py", line 24, in run
    return Gen.reV_run(
  File "/usr/local/lib/python3.9/dist-packages/reV/generation/generation.py", line 563, in reV_run
    raise e
  File "/usr/local/lib/python3.9/dist-packages/reV/generation/generation.py", line 558, in reV_run
    gen._parallel_run(max_workers=max_workers, pool_size=pool_size,
  File "/usr/local/lib/python3.9/dist-packages/reV/generation/base.py", line 1168, in _parallel_run
    result = future.result(timeout=timeout)
  File "/usr/lib/python3.9/concurrent/futures/_base.py", line 440, in result
    return self.__get_result()
  File "/usr/lib/python3.9/concurrent/futures/_base.py", line 389, in __get_result
    raise self._exception
reV.utilities.exceptions.SAMExecutionError: PySAM raised an error while executing: "pvwattsv7" for site 0

Code Sample

test.py:

#!/usr/bin/env python
import numpy as np
from reV.config.project_points import ProjectPoints
from reV.generation.generation import Gen

TECHNOLOGY = "pvwattsv7"
COORDINATES = np.array([
    [55.75, 356.25], [53.5, 357.25], [51.5, 0.0], [52.0, 356.75]
])
RESOURCE_PATH = "/appdata/solar.h5"
SAM_CONFIG_PATH = "/appdata/sam_config_solar_fixed.json"
OUTPUT_REQ = ("cf_mean", "cf_profile")
OUTPUT_PATH = "/appdata/out/rev-solar-test"

def create_pp():
    return ProjectPoints.lat_lon_coords(
        COORDINATES,
        RESOURCE_PATH,
        SAM_CONFIG_PATH
    )

def run():
    return Gen.reV_run(
        TECHNOLOGY,
        create_pp(),
        SAM_CONFIG_PATH,
        RESOURCE_PATH,
        max_workers=4,
        out_fpath=OUTPUT_PATH,
        output_request=OUTPUT_REQ
    )

if __name__ == "__main__":
    data = run()
    print(data)

We ran this in a Docker container (based on debian:11), see Dockerfile below:

FROM debian:11

WORKDIR /app

VOLUME ["/appdata"]

RUN apt-get update && \
    apt-get install -y python3 python3-pip git

RUN git clone https://github.com/NREL/reV.git --branch v0.6.4 --depth 1 rev

RUN cd rev && \
    pip3 install .

COPY test.py /app/test.py

CMD ["python3", "test.py"]

SAM config (sam_config_solar_fixed.json):

{
    "azimuth": 180,
    "array_type": 0,
    "dc_ac_ratio": 1.2,
    "system_capacity": 99999.509328000015,
    "clearsky": false,
    "compute_module": "pvwattsv7",
    "gcr": 0.50,
    "inv_eff": 98.271,
    "losses": 11.179,
    "module_type": 1,
    "tilt": 0
 }

To Reproduce Steps to reproduce the problem behavior

  1. Copy the Dockerfile from the definition above
  2. Copy the example code test.py from the definition above
  3. Build the Docker image (docker build -t rev-test .)
  4. Run the docker image: 
    $ docker run --rm -it \
-v "$PWD/sam_config_solar_fixed.json:/appdata/sam_config_solar_fixed.json" \
-v "$PWD/solar.h5:/appdata/solar.h5" \
-v "$PWD/test.py:/app/test.py" \
rev-test

Expected behavior For the program to execute successfully. The SAM config has been run with the example dataset for solar (brazil_solar.h5), which worked well (finished without errors)

Screenshots N/A

System (please complete the following information):

Additional context

grantbuster commented 2 years ago

Hey thank you for the detailed bug report @netr0m. It's highly likely that this error is due to an error with your custom resource file. Here are some small sample resource files that you can check formatting against: https://github.com/NREL/reV/blob/main/tests/data/nsrdb/ri_100_nsrdb_2013.h5

Here are some other things i would recommend checking / trying:

  1. Write your h5 file with the rex Outputs utility, this will help ensure that the data has the reV-compliant format (especially the meta and time index datasets).
  2. Make sure the meta data being sent to the Outputs write methods is a DataFrame where every row is a pixel and these columns are present: latitude, longitude, timezone, elevation
  3. Make sure the time index and associated temporal data is in UTC
  4. Make sure DNI is not zero. SAM likes DNI and DHI as inputs. You can look up the equation to calculate DNI from GHI+DHI+SZA.
  5. Make sure latitude is between -90 and 90, longitude is between -180 and 180.

If you figure it out, be sure to let me know what went wrong so we can make the docstrings more clear.

netr0m commented 1 year ago

Hi Grant,

Thanks for the quick response! We had some other projects taking up time in the meanwhile, but we've finally managed to do a successful run!

We initially believed that the issue was with our conversion of the data for calculating dni, dhi and ghi. We focused our efforts on those conversions (we were using the ERA5 climate data set as our basis, extracting ssrd and fdir which we in turn converted to the various irradiation types.

The final fix that resolved our issue was the fact that the latitudes and longitudes were not in the range -90..90 and -180..180. I suppose that this, in combination with our previously incorrect calculation of the irradiation values, were the cause of the failed runs.

Thanks again!

grantbuster commented 1 year ago

Ah yeah that makes sense. The SAM PVWatts module needs the lat/lon to calculate solar zenith angle to get the plane of array irradiance. Thanks for letting us know what the problem is! We'll have to add a check on the lat/lon range at some point. I'll add a feature request.