`inclusion_weights` argument isn't valid in JSON

nickwg03 commented 4 years ago

Bug Description The inclusion_weights argument as specified as a python dictionary works fine, but numbers can't be used as keys in JSON. Therefore, I'm not able to make this configuration valid within the JSON configs.

Example of dictionary implementation: https://github.com/NREL/reV/blob/988486ea2ea9854ea86109378d93ebcf00887526/tests/test_supply_curve_exclusions.py#L295

Code Sample This is the closest I can get to making it work in JSON, but the string number IDs break reV as expected:

        "mrlc_nlcd_2016": {
            "inclusion_weights": {
                "11": 0,
                "90": 0,
                "95": 0,
                "41": 0.5,
                "42": 0.5,
                "43": 0.5
            },
            "exclude_nodata": false
        },

To Reproduce Steps to reproduce the problem behavior

Go to '...'
Click on '....'
Scroll down to '....'
See error

Expected behavior JSON config should allow for inputting inclusion_weights argument somehow that makes sense. Maybe the string IDs get converted to integers within reV.

Charge code GMLC.10246.09.05.01

MRossol commented 4 years ago

@nickwg03, I'm confused as to why this doesn't work since we already convert the key from a string here: https://github.com/NREL/reV/blob/e0cf2d5a819cabcd2ecff2bd38c4364778b02d96/reV/supply_curve/exclusions.py#L367-L368

Have you tried running it from the config? If so what error did you get?

grantbuster commented 4 years ago

@MRossol could it be that we have to convert to an int?

MRossol commented 4 years ago

Possibly, depends on how picky np.isin is: https://github.com/NREL/reV/blob/e0cf2d5a819cabcd2ecff2bd38c4364778b02d96/reV/supply_curve/exclusions.py#L304

Hence wanting @nickwg03's traceback

nickwg03 commented 4 years ago

concurrent.futures.process._RemoteTraceback:
"""
Traceback (most recent call last):
  File "/home/ngrue/.conda-envs/conda-prod/lib/python3.8/concurrent/futures/process.py", line 239, in _process_worker
    r = call_item.fn(*call_item.args, **call_item.kwargs)
  File "/home/ngrue/.conda-envs/conda-prod/lib/python3.8/site-packages/reV/supply_curve/sc_aggregation.py", line 998, in run_serial
    logger.debug('Serial aggregation: '
  File "/home/ngrue/.conda-envs/conda-prod/lib/python3.8/site-packages/reV/supply_curve/sc_aggregation.py", line 961, in run_serial
    pointsum = SupplyCurvePointSummary.summarize(
  File "/home/ngrue/.conda-envs/conda-prod/lib/python3.8/site-packages/reV/supply_curve/point_summary.py", line 688, in summarize
    with cls(gid, excl_fpath, gen_fpath, tm_dset, gen_index,
  File "/home/ngrue/.conda-envs/conda-prod/lib/python3.8/site-packages/reV/supply_curve/point_summary.py", line 108, in __init__
    super().__init__(gid, excl, gen, tm_dset, gen_index,
  File "/home/ngrue/.conda-envs/conda-prod/lib/python3.8/site-packages/reV/supply_curve/points.py", line 969, in __init__
    super().__init__(gid, excl, gen, tm_dset,
  File "/home/ngrue/.conda-envs/conda-prod/lib/python3.8/site-packages/reV/supply_curve/points.py", line 623, in __init__
    super().__init__(gid, excl, tm_dset, excl_dict=excl_dict,
  File "/home/ngrue/.conda-envs/conda-prod/lib/python3.8/site-packages/reV/supply_curve/points.py", line 204, in __init__
    self._check_excl()
  File "/home/ngrue/.conda-envs/conda-prod/lib/python3.8/site-packages/reV/supply_curve/points.py", line 422, in _check_excl
    if all(self.excl_data_flat[self.bool_mask] == 0):
  File "/home/ngrue/.conda-envs/conda-prod/lib/python3.8/site-packages/reV/supply_curve/points.py", line 391, in excl_data_flat
    self._excl_data_flat = self.excl_data.flatten()
  File "/home/ngrue/.conda-envs/conda-prod/lib/python3.8/site-packages/reV/supply_curve/points.py", line 365, in excl_data
    self._excl_data = self.exclusions[self.rows, self.cols]
  File "/home/ngrue/.conda-envs/conda-prod/lib/python3.8/site-packages/reV/supply_curve/exclusions.py", line 468, in __getitem__
    return self._generate_mask(*ds_slice)
  File "/home/ngrue/.conda-envs/conda-prod/lib/python3.8/site-packages/reV/supply_curve/exclusions.py", line 790, in _generate_mask
    layer_mask = layer[self.excl_h5[layer_slice]]
  File "/home/ngrue/.conda-envs/conda-prod/lib/python3.8/site-packages/reV/supply_curve/exclusions.py", line 92, in __getitem__
    return self._apply_mask(data)
  File "/home/ngrue/.conda-envs/conda-prod/lib/python3.8/site-packages/reV/supply_curve/exclusions.py", line 214, in _apply_mask
    data = func(data)
  File "/home/ngrue/.conda-envs/conda-prod/lib/python3.8/site-packages/reV/supply_curve/exclusions.py", line 373, in _weights_mask
    mask += self._value_mask(data, [value], include=True) * weight
numpy.core._exceptions.UFuncTypeError: Cannot cast ufunc 'add' output from dtype('float64') to dtype('int64') with casting rule 'same_kind'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ngrue/.conda-envs/conda-prod/lib/python3.8/concurrent/futures/process.py", line 204, in _sendback_result
    result_queue.put(_ResultItem(work_id, result=result,
  File "/home/ngrue/.conda-envs/conda-prod/lib/python3.8/multiprocessing/queues.py", line 362, in put
    obj = _ForkingPickler.dumps(obj)
  File "/home/ngrue/.conda-envs/conda-prod/lib/python3.8/multiprocessing/reduction.py", line 51, in dumps
    cls(buf, protocol).dump(obj)
_pickle.PicklingError: Can't pickle <class 'numpy.core._exceptions.UFuncTypeError'>: it's not the same object as numpy.core._exceptions.UFuncTypeError
"""

nickwg03 commented 4 years ago

@MRossol just making sure you saw the above. Seems like it is a float/int issue.

MRossol commented 4 years ago

yup, working on it now

NREL / reV

`inclusion_weights` argument isn't valid in JSON #237