search

PSLmodels / Tax-Brain

Tax-Brain is an integrator model for PSL tax models
http://taxbrain.pslmodels.org/
MIT License
9 stars 14 forks source link

COMP error #61

Closed hdoupe closed 5 years ago

hdoupe commented 5 years ago

Inputs:

{'policy': {'II_rt7': [{'year': 2019, 'value': 0.45}]}, 'behavior': {'sub': [{'value': 0.25}]}}

Traceback:

Traceback (most recent call last):
  File "/home/distributed/api/celery_app/__init__.py", line 71, in f
    outputs = func(*args, **kwargs)
  File "/home/distributed/api/celery_app/pslmodels_taxbrain_tasks.py", line 30, in sim
    return compconfig.run_model(meta_param_dict, adjustment)
  File "/opt/conda/lib/python3.7/site-packages/compconfig/functions.py", line 165, in run_model
    tb.run()
  File "/opt/conda/lib/python3.7/site-packages/taxbrain-2.2.1-py3.7.egg/taxbrain/taxbrain.py", line 132, in run
    self._dynamic_run(varlist)
  File "/opt/conda/lib/python3.7/site-packages/taxbrain-2.2.1-py3.7.egg/taxbrain/taxbrain.py", line 293, in _dynamic_run
    dump=True)
  File "/opt/conda/lib/python3.7/site-packages/behresp-0.9.0-py3.7.egg/behresp/behavior.py", line 218, in response
    df1.drop('mtr_inctax', axis='columns', inplace=True)
  File "/opt/conda/lib/python3.7/site-packages/pandas/core/frame.py", line 3940, in drop
    errors=errors)
  File "/opt/conda/lib/python3.7/site-packages/pandas/core/generic.py", line 3780, in drop
    obj = obj._drop_axis(labels, axis, level=level, errors=errors)
  File "/opt/conda/lib/python3.7/site-packages/pandas/core/generic.py", line 3813, in _drop_axis
    result = self.reindex(**{axis_name: new_axis})
  File "/opt/conda/lib/python3.7/site-packages/pandas/util/_decorators.py", line 197, in wrapper
    return func(*args, **kwargs)
  File "/opt/conda/lib/python3.7/site-packages/pandas/core/frame.py", line 3809, in reindex
    return super(DataFrame, self).reindex(**kwargs)
  File "/opt/conda/lib/python3.7/site-packages/pandas/core/generic.py", line 4356, in reindex
    fill_value, copy).__finalize__(self)
  File "/opt/conda/lib/python3.7/site-packages/pandas/core/frame.py", line 3736, in _reindex_axes
    fill_value, limit, tolerance)
  File "/opt/conda/lib/python3.7/site-packages/pandas/core/frame.py", line 3761, in _reindex_columns
    allow_dups=False)
  File "/opt/conda/lib/python3.7/site-packages/pandas/core/generic.py", line 4490, in _reindex_with_indexers
    copy=copy)
  File "/opt/conda/lib/python3.7/site-packages/pandas/core/internals/managers.py", line 1231, in reindex_indexer
    fill_tuple=(fill_value,))
  File "/opt/conda/lib/python3.7/site-packages/pandas/core/internals/managers.py", line 1312, in _slice_take_blocks_ax0
    fill_tuple=None))
  File "/opt/conda/lib/python3.7/site-packages/pandas/core/internals/blocks.py", line 1234, in take_nd
    allow_fill=False, fill_value=fill_value)
  File "/opt/conda/lib/python3.7/site-packages/pandas/core/algorithms.py", line 1651, in take_nd
    out = np.empty(out_shape, dtype=dtype)
MemoryError
hdoupe commented 5 years ago

I'll bump the server size. On the Tax-Brain side, Behavior-Response copies the calculator objects which doubles the memory usage for each run.

andersonfrailey commented 5 years ago

Thanks for the report, @hdoupe. I think this memory error is also why we have trouble with Travis. I'll see if we can find a way to reduce memory usage.

hdoupe commented 5 years ago

No problem, I bumped the server memory to 6 CPU and 6 RAM. One of the Behavior sims I just ran took 400 seconds. It may be worth bumping the expected run time to 400 secs.