search

NREL / reV

Renewable Energy Potential (reV) Model
https://nrel.github.io/reV/
BSD 3-Clause "New" or "Revised" License
107 stars 24 forks source link

5min solar downscale not currently working #293

Closed EvanRosenliebNREL closed 3 years ago

EvanRosenliebNREL commented 3 years ago

Bug Description "downscale": "5min" in the sam config for a solar rev run leads to an stdout error and output h5's that are populated with all 0's

Full Traceback

/home/erosenli/miniconda3/envs/nsrdb/lib/python3.7/site-packages/pandas/core/indexing.py:376: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  self.obj[key] = _infer_fill_value(value)
/home/erosenli/miniconda3/envs/nsrdb/lib/python3.7/site-packages/pandas/core/indexing.py:494: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  self.obj[item] = s
concurrent.futures.process._RemoteTraceback:
"""
Traceback (most recent call last):
  File "/home/erosenli/miniconda3/envs/nsrdb/lib/python3.7/site-packages/pandas/tseries/frequencies.py", line 148, in to_offset
    splitted = re.split(libfreqs.opattern, freq)
  File "/home/erosenli/miniconda3/envs/nsrdb/lib/python3.7/re.py", line 215, in split
    return _compile(pattern, flags).split(string, maxsplit)
TypeError: expected string or bytes-like object

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/erosenli/miniconda3/envs/nsrdb/lib/python3.7/concurrent/futures/process.py", line 239, in _process_worker
    r = call_item.fn(*call_item.args, **call_item.kwargs)
  File "/home/erosenli/miniconda3/envs/nsrdb/lib/python3.7/site-packages/reV/generation/generation.py", line 420, in run
    raise e
  File "/home/erosenli/miniconda3/envs/nsrdb/lib/python3.7/site-packages/reV/generation/generation.py", line 416, in run
    output_request=output_request)
  File "/home/erosenli/miniconda3/envs/nsrdb/lib/python3.7/site-packages/reV/SAM/generation.py", line 308, in reV_run
    output_request=output_request)
/home/erosenli/miniconda3/envs/nsrdb/lib/python3.7/site-packages/pandas/core/indexing.py:376: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  self.obj[key] = _infer_fill_value(value)
/home/erosenli/miniconda3/envs/nsrdb/lib/python3.7/site-packages/pandas/core/indexing.py:494: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  self.obj[item] = s
concurrent.futures.process._RemoteTraceback:
"""
Traceback (most recent call last):
  File "/home/erosenli/miniconda3/envs/nsrdb/lib/python3.7/site-packages/pandas/tseries/frequencies.py", line 148, in to_offset
    splitted = re.split(libfreqs.opattern, freq)
  File "/home/erosenli/miniconda3/envs/nsrdb/lib/python3.7/re.py", line 215, in split
    return _compile(pattern, flags).split(string, maxsplit)
TypeError: expected string or bytes-like object

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/erosenli/miniconda3/envs/nsrdb/lib/python3.7/concurrent/futures/process.py", line 239, in _process_worker
    r = call_item.fn(*call_item.args, **call_item.kwargs)
  File "/home/erosenli/miniconda3/envs/nsrdb/lib/python3.7/site-packages/reV/generation/generation.py", line 420, in run
    raise e
  File "/home/erosenli/miniconda3/envs/nsrdb/lib/python3.7/site-packages/reV/generation/generation.py", line 416, in run
    output_request=output_request)
  File "/home/erosenli/miniconda3/envs/nsrdb/lib/python3.7/site-packages/reV/SAM/generation.py", line 308, in reV_run
    output_request=output_request)
  File "/home/erosenli/miniconda3/envs/nsrdb/lib/python3.7/site-packages/reV/SAM/SAM.py", line 540, in get_sam_res
    return SamResourceRetriever.get(*args, **kwargs)
  File "/home/erosenli/miniconda3/envs/nsrdb/lib/python3.7/site-packages/reV/SAM/SAM.py", line 238, in get
    res = res_handler.preload_SAM(res_file, *args, **kwargs)
  File "/home/erosenli/miniconda3/envs/nsrdb/lib/python3.7/site-packages/rex/renewable_resource.py", line 295, in preload_SAM
    bifacial=bifacial, downscale=downscale)
  File "/home/erosenli/miniconda3/envs/nsrdb/lib/python3.7/site-packages/rex/renewable_resource.py", line 295, in preload_SAM
    bifacial=bifacial, downscale=downscale)
  File "/home/erosenli/miniconda3/envs/nsrdb/lib/python3.7/site-packages/rex/renewable_resource.py", line 238, in _preload_SAM
    sam_vars=SAM_res.var_list)
  File "/home/erosenli/miniconda3/envs/nsrdb/lib/python3.7/site-packages/rex/utilities/downscale.py", line 127, in downscale_nsrdb
    time_index = make_time_index(res.time_index.year[0], frequency)
  File "/home/erosenli/miniconda3/envs/nsrdb/lib/python3.7/site-packages/rex/utilities/downscale.py", line 38, in make_time_index
    freq=frequency)[:-1]
  File "/home/erosenli/miniconda3/envs/nsrdb/lib/python3.7/site-packages/pandas/core/indexes/datetimes.py", line 1585, in date_range
    **kwargs
  File "/home/erosenli/miniconda3/envs/nsrdb/lib/python3.7/site-packages/pandas/core/arrays/datetimes.py", line 488, in _generate_range
    freq = to_offset(freq)
  File "/home/erosenli/miniconda3/envs/nsrdb/lib/python3.7/site-packages/pandas/tseries/frequencies.py", line 174, in to_offset
    raise ValueError(libfreqs.INVALID_FREQ_ERR_MSG.format(freq))
ValueError: Invalid frequency: {'frequency': '5min'}
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/erosenli/miniconda3/envs/nsrdb/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/erosenli/miniconda3/envs/nsrdb/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/erosenli/miniconda3/envs/nsrdb/lib/python3.7/site-packages/reV/generation/cli_gen.py", line 752, in <module>
    main(obj={})
  File "/home/erosenli/miniconda3/envs/nsrdb/lib/python3.7/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/home/erosenli/miniconda3/envs/nsrdb/lib/python3.7/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/home/erosenli/miniconda3/envs/nsrdb/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/erosenli/miniconda3/envs/nsrdb/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/erosenli/miniconda3/envs/nsrdb/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/erosenli/miniconda3/envs/nsrdb/lib/python3.7/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/erosenli/miniconda3/envs/nsrdb/lib/python3.7/site-packages/click/decorators.py", line 21, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/erosenli/miniconda3/envs/nsrdb/lib/python3.7/site-packages/reV/generation/cli_gen.py", line 436, in local
    timeout=timeout)
  File "/home/erosenli/miniconda3/envs/nsrdb/lib/python3.7/site-packages/reV/generation/generation.py", line 585, in reV_run
    raise e
  File "/home/erosenli/miniconda3/envs/nsrdb/lib/python3.7/site-packages/reV/generation/generation.py", line 581, in reV_run
    timeout=timeout, **kwargs)
  File "/home/erosenli/miniconda3/envs/nsrdb/lib/python3.7/site-packages/reV/generation/base.py", line 1146, in _parallel_run
    result = future.result(timeout=timeout)
  File "/home/erosenli/miniconda3/envs/nsrdb/lib/python3.7/concurrent/futures/_base.py", line 435, in result
    return self.__get_result()
  File "/home/erosenli/miniconda3/envs/nsrdb/lib/python3.7/concurrent/futures/_base.py", line 384, in __get_result
    raise self._exception
ValueError: Invalid frequency: {'frequency': '5min'}

Code Sample Here is the sam config:

{
    "azimuth": 180,
    "array_type": 2,
    "capital_cost": 39767200,
    "clearsky": false,
    "compute_module": "pvwattsv5",
    "dc_ac_ratio": 1.3,
    "fixed_charge_rate": 0.096,
    "fixed_operating_cost": 260000,
    "gcr": 0.4,
    "inv_eff": 96,
    "losses": 14.07566,
    "module_type": 0,
    "system_capacity": 20000,
    "tilt": 0,
    "variable_operating_cost": 0,
    "downscale": "5min"
}

To Reproduce Run reV generation using the gen config in /projects/rev/new_projects/reeds_solar_5min

Expected behavior h5's to be generated with cf_profile at five minute time resolution that is not populated with 0's, preferable with no stdout error.

System (please complete the following information):

nrel-pysam                2.1.4                    pypi_0    pypi
nrel-pysam-stubs          2.2.2                    pypi_0    pypi
nrel-rev                  0.4.38                   pypi_0    pypi
nrel-rex                  0.2.30                   pypi_0    pypi
nsrdb                     3.2.0                     dev_0    <develop>

Charge code Currently looking into whether there is budget left as I have already gone past estimated time for this task

grantbuster commented 3 years ago

@EvanRosenliebNREL, you're still on old versions of reV and rex. See reV version 0.4.55 and rex version 0.2.52. I can say with certainty that this is at least part of the issue, especially since the stack trace is showing errors on lines that no longer exist in the reV generation module.

WilliamsTravis commented 3 years ago

@grantbuster When we install nsrdb it is automatically downgrading our reV installations to 0.4.38. It could be the case that the "downscale" key is still in the generation config. Is it necessary to use an old reV version with the nsrdb (we assumed it was), or is this a bug?

MRossol commented 3 years ago

how are you installing nsrdb? From enterprise or github.com?

EvanRosenliebNREL commented 3 years ago

Github.

EDIT sorry Enterprise.

grantbuster commented 3 years ago

so the NSRDB requires reV >= 0.4.37 but I wouldnt recommend using reV 0.4.37 regardless. I think this is pip behavior. Just run pip install nrel-rev --upgrade and same for rex.

grantbuster commented 3 years ago

Just because NSRDB wants reV >= 0.4.37 doesnt mean thats the version of reV YOU should be using. It's just a minimum requirement for us to run the NSRDB software (not the reV downscaling stuff).

EvanRosenliebNREL commented 3 years ago

Okay -- I have jobs in the queue if they get through without the error I will close.

WilliamsTravis commented 3 years ago

@grantbuster Yeah i saw that in the setup.py. However, neither of us purposely downgraded, it was automatic. Not sure why it would do that with the given install_requires entry. Anyways, that's cool, we can upgrade back no problem.

MRossol commented 3 years ago

@WilliamsTravis, Thats super weird. I've never seen pip downgrade when you have a >= requirement. Can you post the full traceback during the pip-install? There might be another dependency that has reV or rex pinned at 0.4.38...

WilliamsTravis commented 3 years ago

Yeah, i thought it was weird too! Let me reinstall and get that to you.

EvanRosenliebNREL commented 3 years ago 
(rev) [erosenli@el1 nsrdb]$ pip install -e .
Obtaining file:///home/erosenli/nsrdb
Requirement already satisfied: numpy>=1.16 in /home/erosenli/miniconda3/envs/rev/lib/python3.7/site-packages (from nsrdb==3.2.0) (1.19.2)
Collecting pandas<1,>=0.25
  Using cached pandas-0.25.3-cp37-cp37m-manylinux1_x86_64.whl (10.4 MB)
Requirement already satisfied: click>=7.0 in /home/erosenli/miniconda3/envs/rev/lib/python3.7/site-packages (from nsrdb==3.2.0) (7.1.2)
Requirement already satisfied: scipy>=1.3 in /home/erosenli/miniconda3/envs/rev/lib/python3.7/site-packages (from nsrdb==3.2.0) (1.6.2)
Requirement already satisfied: pyhdf in /home/erosenli/miniconda3/envs/rev/lib/python3.7/site-packages (from nsrdb==3.2.0) (0.10.2)
Collecting h5py>=3.1.0
  Using cached h5py-3.2.1-cp37-cp37m-manylinux1_x86_64.whl (4.1 MB)
Collecting scikit-learn>=0.21
  Using cached scikit_learn-0.24.1-cp37-cp37m-manylinux2010_x86_64.whl (22.3 MB)
Requirement already satisfied: netcdf4>=1.4 in /home/erosenli/miniconda3/envs/rev/lib/python3.7/site-packages (from nsrdb==3.2.0) (1.5.6)
Requirement already satisfied: matplotlib>=3.1 in /home/erosenli/miniconda3/envs/rev/lib/python3.7/site-packages (from nsrdb==3.2.0) (3.3.4)
Collecting pytest>=5.2
  Using cached pytest-6.2.3-py3-none-any.whl (280 kB)
Collecting ipython
  Using cached ipython-7.22.0-py3-none-any.whl (785 kB)
Collecting notebook
  Using cached notebook-6.3.0-py3-none-any.whl (9.5 MB)
Requirement already satisfied: psutil in /home/erosenli/miniconda3/envs/rev/lib/python3.7/site-packages (from nsrdb==3.2.0) (5.8.0)
Collecting pre-commit
  Downloading pre_commit-2.12.1-py2.py3-none-any.whl (189 kB)
     |████████████████████████████████| 189 kB 1.9 MB/s
Collecting flake8
  Downloading flake8-3.9.1-py2.py3-none-any.whl (73 kB)
     |████████████████████████████████| 73 kB 804 kB/s
Collecting pylint
  Using cached pylint-2.7.4-py3-none-any.whl (346 kB)
Requirement already satisfied: NREL-rex>=0.2.30 in /home/erosenli/miniconda3/envs/rev/lib/python3.7/site-packages (from nsrdb==3.2.0) (0.2.52)
Requirement already satisfied: NREL-reV>=0.4.37 in /home/erosenli/miniconda3/envs/rev/lib/python3.7/site-packages (from nsrdb==3.2.0) (0.4.55)
Collecting cached-property
  Using cached cached_property-1.5.2-py2.py3-none-any.whl (7.6 kB)
Requirement already satisfied: cycler>=0.10 in /home/erosenli/miniconda3/envs/rev/lib/python3.7/site-packages (from matplotlib>=3.1->nsrdb==3.2.0) (0.10.0)
Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.3 in /home/erosenli/miniconda3/envs/rev/lib/python3.7/site-packages (from matplotlib>=3.1->nsrdb==3.2.0) (2.4.7)
Requirement already satisfied: kiwisolver>=1.0.1 in /home/erosenli/miniconda3/envs/rev/lib/python3.7/site-packages (from matplotlib>=3.1->nsrdb==3.2.0) (1.3.1)
Requirement already satisfied: pillow>=6.2.0 in /home/erosenli/miniconda3/envs/rev/lib/python3.7/site-packages (from matplotlib>=3.1->nsrdb==3.2.0) (8.2.0)
Requirement already satisfied: python-dateutil>=2.1 in /home/erosenli/miniconda3/envs/rev/lib/python3.7/site-packages (from matplotlib>=3.1->nsrdb==3.2.0) (2.8.1)
Requirement already satisfied: six in /home/erosenli/miniconda3/envs/rev/lib/python3.7/site-packages (from cycler>=0.10->matplotlib>=3.1->nsrdb==3.2.0) (1.15.0)
Requirement already satisfied: cftime in /home/erosenli/miniconda3/envs/rev/lib/python3.7/site-packages (from netcdf4>=1.4->nsrdb==3.2.0) (1.4.1)
Requirement already satisfied: NREL-PySAM==2.1.4 in /home/erosenli/miniconda3/envs/rev/lib/python3.7/site-packages (from NREL-reV>=0.4.37->nsrdb==3.2.0) (2.1.4)
Requirement already satisfied: packaging>=20.3 in /home/erosenli/miniconda3/envs/rev/lib/python3.7/site-packages (from NREL-reV>=0.4.37->nsrdb==3.2.0) (20.9)
Requirement already satisfied: plotly>=4.7.1 in /home/erosenli/miniconda3/envs/rev/lib/python3.7/site-packages (from NREL-reV>=0.4.37->nsrdb==3.2.0) (4.14.3)
Requirement already satisfied: plotting>=0.0.6 in /home/erosenli/miniconda3/envs/rev/lib/python3.7/site-packages (from NREL-reV>=0.4.37->nsrdb==3.2.0) (0.0.7)
Requirement already satisfied: NREL-PySAM-stubs in /home/erosenli/miniconda3/envs/rev/lib/python3.7/site-packages (from NREL-PySAM==2.1.4->NREL-reV>=0.4.37->nsrdb==3.2.0) (2.2.2)
Requirement already satisfied: h5pyd>=0.7.0 in /home/erosenli/miniconda3/envs/rev/lib/python3.7/site-packages (from NREL-rex>=0.2.30->nsrdb==3.2.0) (0.8.3)
INFO: pip is looking at multiple versions of nrel-pysam to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of nrel-rev to determine which version is compatible with other requirements. This could take a while.
Collecting NREL-reV>=0.4.37
  Using cached NREL_reV-0.4.54-py3-none-any.whl (848 kB)
  Using cached NREL_reV-0.4.53-py3-none-any.whl (845 kB)
  Using cached NREL_reV-0.4.52-py3-none-any.whl (845 kB)
  Using cached NREL_reV-0.4.51-py3-none-any.whl (845 kB)
  Using cached NREL_reV-0.4.50-py3-none-any.whl (845 kB)
  Using cached NREL_reV-0.4.49-py3-none-any.whl (845 kB)
  Using cached NREL_reV-0.4.48-py3-none-any.whl (845 kB)
  Using cached NREL_reV-0.4.47-py3-none-any.whl (845 kB)
  Using cached NREL_reV-0.4.46-py3-none-any.whl (845 kB)
  Using cached NREL_reV-0.4.45-py3-none-any.whl (844 kB)
  Using cached NREL_reV-0.4.44-py3-none-any.whl (841 kB)
  Using cached NREL_reV-0.4.43-py3-none-any.whl (841 kB)
  Using cached NREL_reV-0.4.42-py3-none-any.whl (841 kB)
  Using cached NREL_reV-0.4.41-py3-none-any.whl (841 kB)
  Using cached NREL_reV-0.4.40-py3-none-any.whl (841 kB)
  Using cached NREL_reV-0.4.39-py3-none-any.whl (833 kB)
  Using cached NREL_reV-0.4.38-py3-none-any.whl (833 kB)
  Using cached NREL_reV-0.4.37-py3-none-any.whl (832 kB)
Collecting NREL-rex>=0.2.30
  Using cached NREL_rex-0.2.51-py3-none-any.whl (114 kB)
  Using cached NREL_rex-0.2.50-py3-none-any.whl (114 kB)
  Using cached NREL_rex-0.2.49-py3-none-any.whl (113 kB)
  Using cached NREL_rex-0.2.48-py3-none-any.whl (113 kB)
  Using cached NREL_rex-0.2.47-py3-none-any.whl (111 kB)
  Using cached NREL_rex-0.2.46-py3-none-any.whl (105 kB)
  Using cached NREL_rex-0.2.45-py3-none-any.whl (105 kB)
  Using cached NREL_rex-0.2.44-py3-none-any.whl (105 kB)
INFO: pip is looking at multiple versions of nrel-pysam to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of nrel-rev to determine which version is compatible with other requirements. This could take a while.
  Using cached NREL_rex-0.2.43-py3-none-any.whl (105 kB)
  Using cached NREL_rex-0.2.42-py3-none-any.whl (104 kB)
INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. If you want to abort this run, you can press Ctrl + C to do so. To improve how pip performs, tell us what happened here: https://pip.pypa.io/surveys/backtracking
INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. If you want to abort this run, you can press Ctrl + C to do so. To improve how pip performs, tell us what happened here: https://pip.pypa.io/surveys/backtracking
  Using cached NREL_rex-0.2.41-py3-none-any.whl (104 kB)
  Using cached NREL_rex-0.2.40-py3-none-any.whl (104 kB)
  Using cached NREL_rex-0.2.39-py3-none-any.whl (104 kB)
  Using cached NREL_rex-0.2.38-py3-none-any.whl (103 kB)
  Using cached NREL_rex-0.2.37-py3-none-any.whl (103 kB)
  Using cached NREL_rex-0.2.36-py3-none-any.whl (101 kB)
  Using cached NREL_rex-0.2.35-py3-none-any.whl (101 kB)
  Using cached NREL_rex-0.2.33-py3-none-any.whl (101 kB)
  Using cached NREL_rex-0.2.32-py3-none-any.whl (99 kB)
  Using cached NREL_rex-0.2.31-py3-none-any.whl (96 kB)
  Using cached NREL_rex-0.2.30-py3-none-any.whl (95 kB)
Requirement already satisfied: toml>=0.10.0 in /home/erosenli/miniconda3/envs/rev/lib/python3.7/site-packages (from NREL-rex>=0.2.30->nsrdb==3.2.0) (0.10.2)
Requirement already satisfied: cryptography in /home/erosenli/miniconda3/envs/rev/lib/python3.7/site-packages (from h5pyd>=0.7.0->NREL-rex>=0.2.30->nsrdb==3.2.0) (3.4.7)
Requirement already satisfied: adal in /home/erosenli/miniconda3/envs/rev/lib/python3.7/site-packages (from h5pyd>=0.7.0->NREL-rex>=0.2.30->nsrdb==3.2.0) (1.2.7)
Requirement already satisfied: google-api-python-client in /home/erosenli/miniconda3/envs/rev/lib/python3.7/site-packages (from h5pyd>=0.7.0->NREL-rex>=0.2.30->nsrdb==3.2.0) (2.2.0)
Requirement already satisfied: google-auth-oauthlib in /home/erosenli/miniconda3/envs/rev/lib/python3.7/site-packages (from h5pyd>=0.7.0->NREL-rex>=0.2.30->nsrdb==3.2.0) (0.4.4)
Requirement already satisfied: pytz in /home/erosenli/miniconda3/envs/rev/lib/python3.7/site-packages (from h5pyd>=0.7.0->NREL-rex>=0.2.30->nsrdb==3.2.0) (2021.1)
Requirement already satisfied: requests in /home/erosenli/miniconda3/envs/rev/lib/python3.7/site-packages (from h5pyd>=0.7.0->NREL-rex>=0.2.30->nsrdb==3.2.0) (2.25.1)
Requirement already satisfied: msrestazure in /home/erosenli/miniconda3/envs/rev/lib/python3.7/site-packages (from h5pyd>=0.7.0->NREL-rex>=0.2.30->nsrdb==3.2.0) (0.6.4)
Requirement already satisfied: pyjwt in /home/erosenli/miniconda3/envs/rev/lib/python3.7/site-packages (from h5pyd>=0.7.0->NREL-rex>=0.2.30->nsrdb==3.2.0) (2.0.1)
Requirement already satisfied: google-auth<2.0dev in /home/erosenli/miniconda3/envs/rev/lib/python3.7/site-packages (from h5pyd>=0.7.0->NREL-rex>=0.2.30->nsrdb==3.2.0) (1.28.1)
Requirement already satisfied: pyasn1-modules>=0.2.1 in /home/erosenli/miniconda3/envs/rev/lib/python3.7/site-packages (from google-auth<2.0dev->h5pyd>=0.7.0->NREL-rex>=0.2.30->nsrdb==3.2.0) (0.2.8)
Requirement already satisfied: rsa<5,>=3.1.4 in /home/erosenli/miniconda3/envs/rev/lib/python3.7/site-packages (from google-auth<2.0dev->h5pyd>=0.7.0->NREL-rex>=0.2.30->nsrdb==3.2.0) (4.7.2)
Requirement already satisfied: setuptools>=40.3.0 in /home/erosenli/miniconda3/envs/rev/lib/python3.7/site-packages (from google-auth<2.0dev->h5pyd>=0.7.0->NREL-rex>=0.2.30->nsrdb==3.2.0) (52.0.0.post20210125)
Requirement already satisfied: cachetools<5.0,>=2.0.0 in /home/erosenli/miniconda3/envs/rev/lib/python3.7/site-packages (from google-auth<2.0dev->h5pyd>=0.7.0->NREL-rex>=0.2.30->nsrdb==3.2.0) (4.2.1)
Requirement already satisfied: retrying>=1.3.3 in /home/erosenli/miniconda3/envs/rev/lib/python3.7/site-packages (from plotly>=4.7.1->NREL-reV>=0.4.37->nsrdb==3.2.0) (1.3.3)
Requirement already satisfied: seaborn in /home/erosenli/miniconda3/envs/rev/lib/python3.7/site-packages (from plotting>=0.0.6->NREL-reV>=0.4.37->nsrdb==3.2.0) (0.11.1)
Requirement already satisfied: pyasn1<0.5.0,>=0.4.6 in /home/erosenli/miniconda3/envs/rev/lib/python3.7/site-packages (from pyasn1-modules>=0.2.1->google-auth<2.0dev->h5pyd>=0.7.0->NREL-rex>=0.2.30->nsrdb==3.2.0) (0.4.8)
Collecting attrs>=19.2.0
  Using cached attrs-20.3.0-py2.py3-none-any.whl (49 kB)
Collecting importlib-metadata>=0.12
  Downloading importlib_metadata-4.0.0-py3-none-any.whl (16 kB)
Collecting iniconfig
  Using cached iniconfig-1.1.1-py2.py3-none-any.whl (5.0 kB)
Collecting pluggy<1.0.0a1,>=0.12
  Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting py>=1.8.2
  Using cached py-1.10.0-py2.py3-none-any.whl (97 kB)
Collecting zipp>=0.5
  Using cached zipp-3.4.1-py3-none-any.whl (5.2 kB)
Collecting typing-extensions>=3.6.4
  Using cached typing_extensions-3.7.4.3-py3-none-any.whl (22 kB)
Collecting threadpoolctl>=2.0.0
  Using cached threadpoolctl-2.1.0-py3-none-any.whl (12 kB)
Collecting joblib>=0.11
  Using cached joblib-1.0.1-py3-none-any.whl (303 kB)
Requirement already satisfied: cffi>=1.12 in /home/erosenli/miniconda3/envs/rev/lib/python3.7/site-packages (from cryptography->h5pyd>=0.7.0->NREL-rex>=0.2.30->nsrdb==3.2.0) (1.14.5)
Requirement already satisfied: pycparser in /home/erosenli/miniconda3/envs/rev/lib/python3.7/site-packages (from cffi>=1.12->cryptography->h5pyd>=0.7.0->NREL-rex>=0.2.30->nsrdb==3.2.0) (2.20)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/erosenli/miniconda3/envs/rev/lib/python3.7/site-packages (from requests->h5pyd>=0.7.0->NREL-rex>=0.2.30->nsrdb==3.2.0) (1.26.4)
Requirement already satisfied: chardet<5,>=3.0.2 in /home/erosenli/miniconda3/envs/rev/lib/python3.7/site-packages (from requests->h5pyd>=0.7.0->NREL-rex>=0.2.30->nsrdb==3.2.0) (4.0.0)
Requirement already satisfied: certifi>=2017.4.17 in /home/erosenli/miniconda3/envs/rev/lib/python3.7/site-packages (from requests->h5pyd>=0.7.0->NREL-rex>=0.2.30->nsrdb==3.2.0) (2020.12.5)
Requirement already satisfied: idna<3,>=2.5 in /home/erosenli/miniconda3/envs/rev/lib/python3.7/site-packages (from requests->h5pyd>=0.7.0->NREL-rex>=0.2.30->nsrdb==3.2.0) (2.10)
Collecting mccabe<0.7.0,>=0.6.0
  Using cached mccabe-0.6.1-py2.py3-none-any.whl (8.6 kB)
Collecting pyflakes<2.4.0,>=2.3.0
  Using cached pyflakes-2.3.1-py2.py3-none-any.whl (68 kB)
Collecting pycodestyle<2.8.0,>=2.7.0
  Using cached pycodestyle-2.7.0-py2.py3-none-any.whl (41 kB)
Requirement already satisfied: httplib2<1dev,>=0.15.0 in /home/erosenli/miniconda3/envs/rev/lib/python3.7/site-packages (from google-api-python-client->h5pyd>=0.7.0->NREL-rex>=0.2.30->nsrdb==3.2.0) (0.19.1)
Requirement already satisfied: google-auth-httplib2>=0.1.0 in /home/erosenli/miniconda3/envs/rev/lib/python3.7/site-packages (from google-api-python-client->h5pyd>=0.7.0->NREL-rex>=0.2.30->nsrdb==3.2.0) (0.1.0)
Requirement already satisfied: uritemplate<4dev,>=3.0.0 in /home/erosenli/miniconda3/envs/rev/lib/python3.7/site-packages (from google-api-python-client->h5pyd>=0.7.0->NREL-rex>=0.2.30->nsrdb==3.2.0) (3.0.1)
Requirement already satisfied: google-api-core<2dev,>=1.21.0 in /home/erosenli/miniconda3/envs/rev/lib/python3.7/site-packages (from google-api-python-client->h5pyd>=0.7.0->NREL-rex>=0.2.30->nsrdb==3.2.0) (1.26.3)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in /home/erosenli/miniconda3/envs/rev/lib/python3.7/site-packages (from google-api-core<2dev,>=1.21.0->google-api-python-client->h5pyd>=0.7.0->NREL-rex>=0.2.30->nsrdb==3.2.0) (1.53.0)
Requirement already satisfied: protobuf>=3.12.0 in /home/erosenli/miniconda3/envs/rev/lib/python3.7/site-packages (from google-api-core<2dev,>=1.21.0->google-api-python-client->h5pyd>=0.7.0->NREL-rex>=0.2.30->nsrdb==3.2.0) (3.15.8)
Requirement already satisfied: requests-oauthlib>=0.7.0 in /home/erosenli/miniconda3/envs/rev/lib/python3.7/site-packages (from google-auth-oauthlib->h5pyd>=0.7.0->NREL-rex>=0.2.30->nsrdb==3.2.0) (1.3.0)
Requirement already satisfied: oauthlib>=3.0.0 in /home/erosenli/miniconda3/envs/rev/lib/python3.7/site-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib->h5pyd>=0.7.0->NREL-rex>=0.2.30->nsrdb==3.2.0) (3.1.0)
Collecting prompt-toolkit!=3.0.0,!=3.0.1,<3.1.0,>=2.0.0
  Using cached prompt_toolkit-3.0.18-py3-none-any.whl (367 kB)
Collecting pickleshare
  Using cached pickleshare-0.7.5-py2.py3-none-any.whl (6.9 kB)
Collecting traitlets>=4.2
  Using cached traitlets-5.0.5-py3-none-any.whl (100 kB)
Collecting pexpect>4.3
  Using cached pexpect-4.8.0-py2.py3-none-any.whl (59 kB)
Collecting jedi>=0.16
  Using cached jedi-0.18.0-py2.py3-none-any.whl (1.4 MB)
Collecting backcall
  Using cached backcall-0.2.0-py2.py3-none-any.whl (11 kB)
Collecting pygments
  Using cached Pygments-2.8.1-py3-none-any.whl (983 kB)
Collecting decorator
  Using cached decorator-5.0.7-py3-none-any.whl (8.8 kB)
Collecting parso<0.9.0,>=0.8.0
  Using cached parso-0.8.2-py2.py3-none-any.whl (94 kB)
Collecting ptyprocess>=0.5
  Using cached ptyprocess-0.7.0-py2.py3-none-any.whl (13 kB)
Collecting wcwidth
  Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting ipython-genutils
  Using cached ipython_genutils-0.2.0-py2.py3-none-any.whl (26 kB)
Requirement already satisfied: msrest<2.0.0,>=0.6.0 in /home/erosenli/miniconda3/envs/rev/lib/python3.7/site-packages (from msrestazure->h5pyd>=0.7.0->NREL-rex>=0.2.30->nsrdb==3.2.0) (0.6.21)
Requirement already satisfied: isodate>=0.6.0 in /home/erosenli/miniconda3/envs/rev/lib/python3.7/site-packages (from msrest<2.0.0,>=0.6.0->msrestazure->h5pyd>=0.7.0->NREL-rex>=0.2.30->nsrdb==3.2.0) (0.6.0)
Collecting terminado>=0.8.3
  Using cached terminado-0.9.4-py3-none-any.whl (14 kB)
Collecting jupyter-core>=4.6.1
  Using cached jupyter_core-4.7.1-py3-none-any.whl (82 kB)
Collecting Send2Trash>=1.5.0
  Using cached Send2Trash-1.5.0-py3-none-any.whl (12 kB)
Requirement already satisfied: tornado>=6.1 in /home/erosenli/miniconda3/envs/rev/lib/python3.7/site-packages (from notebook->nsrdb==3.2.0) (6.1)
Collecting prometheus-client
  Using cached prometheus_client-0.10.1-py2.py3-none-any.whl (55 kB)
Collecting nbformat
  Using cached nbformat-5.1.3-py3-none-any.whl (178 kB)
Collecting jinja2
  Using cached Jinja2-2.11.3-py2.py3-none-any.whl (125 kB)
Collecting nbconvert
  Using cached nbconvert-6.0.7-py3-none-any.whl (552 kB)
Collecting argon2-cffi
  Using cached argon2_cffi-20.1.0-cp35-abi3-manylinux1_x86_64.whl (97 kB)
Collecting pyzmq>=17
  Using cached pyzmq-22.0.3-cp37-cp37m-manylinux1_x86_64.whl (1.1 MB)
Collecting jupyter-client>=5.3.4
  Using cached jupyter_client-6.1.12-py3-none-any.whl (112 kB)
Collecting ipykernel
  Using cached ipykernel-5.5.3-py3-none-any.whl (120 kB)
Collecting MarkupSafe>=0.23
  Using cached MarkupSafe-1.1.1-cp37-cp37m-manylinux2010_x86_64.whl (33 kB)
Collecting pandocfilters>=1.4.1
  Using cached pandocfilters-1.4.3-py3-none-any.whl
Collecting bleach
  Using cached bleach-3.3.0-py2.py3-none-any.whl (283 kB)
Collecting nbclient<0.6.0,>=0.5.0
  Using cached nbclient-0.5.3-py3-none-any.whl (82 kB)
Collecting entrypoints>=0.2.2
  Using cached entrypoints-0.3-py2.py3-none-any.whl (11 kB)
Collecting testpath
  Using cached testpath-0.4.4-py2.py3-none-any.whl (163 kB)
Collecting defusedxml
  Using cached defusedxml-0.7.1-py2.py3-none-any.whl (25 kB)
Collecting mistune<2,>=0.8.1
  Using cached mistune-0.8.4-py2.py3-none-any.whl (16 kB)
Collecting jupyterlab-pygments
  Using cached jupyterlab_pygments-0.1.2-py2.py3-none-any.whl (4.6 kB)
Collecting async-generator
  Using cached async_generator-1.10-py3-none-any.whl (18 kB)
Collecting nest-asyncio
  Using cached nest_asyncio-1.5.1-py3-none-any.whl (5.0 kB)
Collecting jsonschema!=2.5.0,>=2.4
  Using cached jsonschema-3.2.0-py2.py3-none-any.whl (56 kB)
Collecting pyrsistent>=0.14.0
  Using cached pyrsistent-0.17.3-cp37-cp37m-linux_x86_64.whl
Collecting webencodings
  Using cached webencodings-0.5.1-py2.py3-none-any.whl (11 kB)
Collecting pyyaml>=5.1
  Using cached PyYAML-5.4.1-cp37-cp37m-manylinux1_x86_64.whl (636 kB)
Collecting virtualenv>=20.0.8
  Using cached virtualenv-20.4.3-py2.py3-none-any.whl (7.2 MB)
Collecting cfgv>=2.0.0
  Using cached cfgv-3.2.0-py2.py3-none-any.whl (7.3 kB)
Collecting nodeenv>=0.11.1
  Using cached nodeenv-1.6.0-py2.py3-none-any.whl (21 kB)
Collecting identify>=1.0.0
  Using cached identify-2.2.3-py2.py3-none-any.whl (98 kB)
Collecting distlib<1,>=0.3.1
  Using cached distlib-0.3.1-py2.py3-none-any.whl (335 kB)
Collecting appdirs<2,>=1.4.3
  Using cached appdirs-1.4.4-py2.py3-none-any.whl (9.6 kB)
Collecting filelock<4,>=3.0.0
  Using cached filelock-3.0.12-py3-none-any.whl (7.6 kB)
Collecting isort<6,>=4.2.5
  Using cached isort-5.8.0-py3-none-any.whl (103 kB)
Collecting astroid<2.7,>=2.5.2
  Using cached astroid-2.5.3-py3-none-any.whl (226 kB)
Collecting lazy-object-proxy>=1.4.0
  Using cached lazy_object_proxy-1.6.0-cp37-cp37m-manylinux1_x86_64.whl (55 kB)
Collecting typed-ast<1.5,>=1.4.0
  Using cached typed_ast-1.4.3-cp37-cp37m-manylinux1_x86_64.whl (743 kB)
Collecting wrapt<1.13,>=1.11
  Using cached wrapt-1.12.1-cp37-cp37m-linux_x86_64.whl
Installing collected packages: zipp, typing-extensions, ipython-genutils, traitlets, pyrsistent, importlib-metadata, attrs, wcwidth, pyzmq, ptyprocess, parso, jupyter-core, jsonschema, webencodings, pygments, prompt-toolkit, pickleshare, pexpect, pandas, nest-asyncio, nbformat, MarkupSafe, jupyter-client, jedi, decorator, cached-property, backcall, async-generator, wrapt, typed-ast, testpath, pandocfilters, nbclient, mistune, lazy-object-proxy, jupyterlab-pygments, jinja2, ipython, h5py, filelock, entrypoints, distlib, defusedxml, bleach, appdirs, virtualenv, threadpoolctl, terminado, Send2Trash, pyyaml, pyflakes, pycodestyle, py, prometheus-client, pluggy, NREL-rex, nodeenv, nbconvert, mccabe, joblib, isort, ipykernel, iniconfig, identify, cfgv, astroid, argon2-cffi, scikit-learn, pytest, pylint, pre-commit, NREL-reV, notebook, flake8, nsrdb
  Attempting uninstall: pandas
    Found existing installation: pandas 1.2.3
    Uninstalling pandas-1.2.3:
      Successfully uninstalled pandas-1.2.3
  Attempting uninstall: h5py
    Found existing installation: h5py 2.10.0
    Uninstalling h5py-2.10.0:
      Successfully uninstalled h5py-2.10.0
  Attempting uninstall: NREL-rex
    Found existing installation: NREL-rex 0.2.52
    Uninstalling NREL-rex-0.2.52:
      Successfully uninstalled NREL-rex-0.2.52
  Attempting uninstall: NREL-reV
    Found existing installation: NREL-reV 0.4.55
    Uninstalling NREL-reV-0.4.55:
      Successfully uninstalled NREL-reV-0.4.55
  Running setup.py develop for nsrdb
Successfully installed MarkupSafe-1.1.1 NREL-reV-0.4.38 NREL-rex-0.2.30 Send2Trash-1.5.0 appdirs-1.4.4 argon2-cffi-20.1.0 astroid-2.5.3 async-generator-1.10 attrs-20.3.0 backcall-0.2.0 bleach-3.3.0 cached-property-1.5.2 cfgv-3.2.0 decorator-5.0.7 defusedxml-0.7.1 distlib-0.3.1 entrypoints-0.3 filelock-3.0.12 flake8-3.9.1 h5py-3.2.1 identify-2.2.3 importlib-metadata-4.0.0 iniconfig-1.1.1 ipykernel-5.5.3 ipython-7.22.0 ipython-genutils-0.2.0 isort-5.8.0 jedi-0.18.0 jinja2-2.11.3 joblib-1.0.1 jsonschema-3.2.0 jupyter-client-6.1.12 jupyter-core-4.7.1 jupyterlab-pygments-0.1.2 lazy-object-proxy-1.6.0 mccabe-0.6.1 mistune-0.8.4 nbclient-0.5.3 nbconvert-6.0.7 nbformat-5.1.3 nest-asyncio-1.5.1 nodeenv-1.6.0 notebook-6.3.0 nsrdb pandas-0.25.3 pandocfilters-1.4.3 parso-0.8.2 pexpect-4.8.0 pickleshare-0.7.5 pluggy-0.13.1 pre-commit-2.12.1 prometheus-client-0.10.1 prompt-toolkit-3.0.18 ptyprocess-0.7.0 py-1.10.0 pycodestyle-2.7.0 pyflakes-2.3.1 pygments-2.8.1 pylint-2.7.4 pyrsistent-0.17.3 pytest-6.2.3 pyyaml-5.4.1 pyzmq-22.0.3 scikit-learn-0.24.1 terminado-0.9.4 testpath-0.4.4 threadpoolctl-2.1.0 traitlets-5.0.5 typed-ast-1.4.3 typing-extensions-3.7.4.3 virtualenv-20.4.3 wcwidth-0.2.5 webencodings-0.5.1 wrapt-1.12.1 zipp-3.4.1
grantbuster commented 3 years ago

ive noticed this behavior with pip recently and I think it might be due to an old pip version or just some change in its intended behavior. pip has been doing something funky recently though where if you set a range or a limit, it will try ALL versions that would satisfy that constraint. And then for seemingly no reason it decides on the oldest version. I'm sure there's a good explanation but i've just fixed it after pip does its thing.

You can see this in the pip log here:


INFO: pip is looking at multiple versions of nrel-rev to determine which version is compatible with other requirements. This could take a while.
Collecting NREL-reV>=0.4.37
  Using cached NREL_reV-0.4.54-py3-none-any.whl (848 kB)
  Using cached NREL_reV-0.4.53-py3-none-any.whl (845 kB)
  Using cached NREL_reV-0.4.52-py3-none-any.whl (845 kB)
  Using cached NREL_reV-0.4.51-py3-none-any.whl (845 kB)
  Using cached NREL_reV-0.4.50-py3-none-any.whl (845 kB)
  Using cached NREL_reV-0.4.49-py3-none-any.whl (845 kB)
  Using cached NREL_reV-0.4.48-py3-none-any.whl (845 kB)
  Using cached NREL_reV-0.4.47-py3-none-any.whl (845 kB)
  Using cached NREL_reV-0.4.46-py3-none-any.whl (845 kB)
  Using cached NREL_reV-0.4.45-py3-none-any.whl (844 kB)
  Using cached NREL_reV-0.4.44-py3-none-any.whl (841 kB)
  Using cached NREL_reV-0.4.43-py3-none-any.whl (841 kB)
  Using cached NREL_reV-0.4.42-py3-none-any.whl (841 kB)
  Using cached NREL_reV-0.4.41-py3-none-any.whl (841 kB)
  Using cached NREL_reV-0.4.40-py3-none-any.whl (841 kB)
  Using cached NREL_reV-0.4.39-py3-none-any.whl (833 kB)
  Using cached NREL_reV-0.4.38-py3-none-any.whl (833 kB)
  Using cached NREL_reV-0.4.37-py3-none-any.whl (832 kB)
Collecting NREL-rex>=0.2.30
  Using cached NREL_rex-0.2.51-py3-none-any.whl (114 kB)
  Using cached NREL_rex-0.2.50-py3-none-any.whl (114 kB)
  Using cached NREL_rex-0.2.49-py3-none-any.whl (113 kB)
  Using cached NREL_rex-0.2.48-py3-none-any.whl (113 kB)
  Using cached NREL_rex-0.2.47-py3-none-any.whl (111 kB)
  Using cached NREL_rex-0.2.46-py3-none-any.whl (105 kB)
  Using cached NREL_rex-0.2.45-py3-none-any.whl (105 kB)
  Using cached NREL_rex-0.2.44-py3-none-any.whl (105 kB)
INFO: pip is looking at multiple versions of nrel-pysam to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of nrel-rev to determine which version is compatible with other requirements. This could take a while.
MRossol commented 3 years ago

@grantbuster why don't we just update the setup.py for NSRDB to the most recent version of reV?

grantbuster commented 3 years ago

We can do that but its not the true requirement (what does nsrdb actually need to run?) and also reV releases too often for me to care enough to keep this up to date.

EvanRosenliebNREL commented 3 years ago

Seems like ceteris paribus pip should try to install the largest working version for anything not the smallest version

EvanRosenliebNREL commented 3 years ago

Grant, if we consider this embedded downscaling as a functionality of nsrdb, then it would appear that it does indeed need a higher version of reV in order to run without error, no?

MRossol commented 3 years ago

Your thinking about it backwards, NSRDB only needs v0.4.38, but you're trying to run the most recent version of reV. This definitely seems like a pip issue. Has anyone tried upgrading pip

EvanRosenliebNREL commented 3 years ago 
(nsrdb) [erosenli@el3 stdout]$ pip install pip --upgrade
Requirement already satisfied: pip in /home/erosenli/miniconda3/envs/nsrdb/lib/python3.7/site-packages (21.0.1)
grantbuster commented 3 years ago

Yes what Michael said. It's a super weird non-open source feature that is shared/owned somewhere between reV and NSRDB. Whenever you run reV you should always use the most recent version. NSRDB requirements should not influence this.

Weird non-tested experimental features don't always work perfectly! Just fix your environment!

I think the pip behavior is possibly due to the new 2020 resolver feature. It seems to be in this open issue: https://github.com/pypa/pip/issues/9215

MRossol commented 3 years ago

This also looks related: https://stackoverflow.com/questions/14617136/why-is-pip-installing-an-old-version-of-my-package It looks like the fix is to run: pip install --no-cache-dir --upgrade -e .

MRossol commented 3 years ago

On a related note you all should run: conda clean --all and watch GB of old packages get removed

EvanRosenliebNREL commented 3 years ago

Okie dokie then, looks like it is currently working with most recent rex and rev versions. Thanks all.