Variance decomposition

[sofyaakimova]

Notebook and py. file with code for variance decomposition were added.

[hmgaudecker]

Just trying to use this in a project where not all factors are present in all periods. At least that is what I assume is the issue without looking at it too deeply. Result:

/home/skanehealth/skillmodels/skillmodels/process_data.py:62: FutureWarning: In a future version of pandas all arguments of MultiIndex.set_levels except for the argument 'levels' will be keyword-only
  df.index = df.index.set_levels(range(len(df.index.levels[level])), level)
Traceback (most recent call last):
  File "plots.py", line 44, in <module>
    fig = plot_pairwise_factor_distributions(
  File "/home/skanehealth/skillmodels/skillmodels/visualize_factor_distributions.py", line 64, in plot_pairwise_factor_distributions
    plot_dict = _plot_grid_2d_3d(states, state_ranges, period, n_points)
  File "/home/skanehealth/skillmodels/skillmodels/visualize_factor_distributions.py", line 250, in _plot_grid_2d_3d
    row = int(re.sub("[^0-9]", "", a))
ValueError: invalid literal for int() with base 10: ''

[hmgaudecker]

(should have put this in #58, not sure whether you want to separate the two PRs in the end)

[hmgaudecker]

Hmmm, maybe the old guy just needs a graphing clinic... This is from code that used to work in the summer:

Traceback (most recent call last):
  File "plots.py", line 45, in <module>
    fig = visualize_transition_equations(
  File "/home/skanehealth/skillmodels/skillmodels/visualize_transition_equations.py", line 138, in visualize_transition_equations
    sns.lineplot(
  File "/home/skanehealth/miniconda/envs/skane-struct-bw/lib/python3.8/site-packages/seaborn/_decorators.py", line 46, in inner_f
    return f(**kwargs)
  File "/home/skanehealth/miniconda/envs/skane-struct-bw/lib/python3.8/site-packages/seaborn/relational.py", line 710, in lineplot
    p.plot(ax, kwargs)
  File "/home/skanehealth/miniconda/envs/skane-struct-bw/lib/python3.8/site-packages/seaborn/relational.py", line 471, in plot
    for sub_vars, sub_data in self.iter_data(grouping_vars, from_comp_data=True):
  File "/home/skanehealth/miniconda/envs/skane-struct-bw/lib/python3.8/site-packages/seaborn/_core.py", line 983, in iter_data
    data = self.comp_data
  File "/home/skanehealth/miniconda/envs/skane-struct-bw/lib/python3.8/site-packages/seaborn/_core.py", line 1057, in comp_data
    comp_col.loc[orig.index] = pd.to_numeric(axis.convert_units(orig))
  File "/home/skanehealth/miniconda/envs/skane-struct-bw/lib/python3.8/site-packages/pandas/core/indexing.py", line 723, in __setitem__
    iloc._setitem_with_indexer(indexer, value, self.name)
  File "/home/skanehealth/miniconda/envs/skane-struct-bw/lib/python3.8/site-packages/pandas/core/indexing.py", line 1732, in _setitem_with_indexer
    self._setitem_single_block(indexer, value, name)
  File "/home/skanehealth/miniconda/envs/skane-struct-bw/lib/python3.8/site-packages/pandas/core/indexing.py", line 1959, in _setitem_single_block
    value = self._align_series(indexer, Series(value))
  File "/home/skanehealth/miniconda/envs/skane-struct-bw/lib/python3.8/site-packages/pandas/core/indexing.py", line 2096, in _align_series
    ser = ser.reindex(obj.axes[0][indexer[0]], copy=True)._values
  File "/home/skanehealth/miniconda/envs/skane-struct-bw/lib/python3.8/site-packages/pandas/core/series.py", line 4580, in reindex
    return super().reindex(index=index, **kwargs)
  File "/home/skanehealth/miniconda/envs/skane-struct-bw/lib/python3.8/site-packages/pandas/core/generic.py", line 4818, in reindex
    return self._reindex_axes(
  File "/home/skanehealth/miniconda/envs/skane-struct-bw/lib/python3.8/site-packages/pandas/core/generic.py", line 4839, in _reindex_axes
    obj = obj._reindex_with_indexers(
  File "/home/skanehealth/miniconda/envs/skane-struct-bw/lib/python3.8/site-packages/pandas/core/generic.py", line 4883, in _reindex_with_indexers
    new_data = new_data.reindex_indexer(
  File "/home/skanehealth/miniconda/envs/skane-struct-bw/lib/python3.8/site-packages/pandas/core/internals/managers.py", line 670, in reindex_indexer
    self.axes[axis]._validate_can_reindex(indexer)
  File "/home/skanehealth/miniconda/envs/skane-struct-bw/lib/python3.8/site-packages/pandas/core/indexes/base.py", line 3785, in _validate_can_reindex
    raise ValueError("cannot reindex from a duplicate axis")
ValueError: cannot reindex from a duplicate axis

[janosg]

The first error is fixed in #58 now.

The second error came from a seaborn update and is fixed in #66

[hmgaudecker]

:point_up:

I just rebased on #66 so the changes here would be clearer again. @sofyaakimova @effieHAN, please delete your local branches and pull the current version.

Assuming you are on some other branch than variance_decomposition:

$ git branch -D variance_decomposition
$ git pull
$ git switch variance_decomposition 

Thanks!

[codecov-commenter]

Codecov Report

:exclamation: No coverage uploaded for pull request base (main@9e7c32a). Click here to learn what that means. The diff coverage is n/a.

@@           Coverage Diff           @@
##             main      #62   +/-   ##
=======================================
  Coverage        ?   93.61%           
=======================================
  Files           ?       32           
  Lines           ?     2491           
  Branches        ?        0           
=======================================
  Hits            ?     2332           
  Misses          ?      159           
  Partials        ?        0           

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 9e7c32a...1e4da2f. Read the comment docs.

[review-notebook-app[bot]]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB