ENH: Implement aggregation from one grouping (e.g. bg) to another (e.g. fg)

iza-institute-of-labor-economics / gettsim

The GErman Taxes and Transfers SIMulator

https://gettsim.readthedocs.io/

GNU Affero General Public License v3.0

54 stars 32 forks source link

ENH: Implement aggregation from one grouping (e.g. bg) to another (e.g. fg) #555

Open ChristianZimpelmann opened 1 year ago

ChristianZimpelmann commented 1 year ago

Is your feature request related to a problem?

Sometime one needs to aggregate a variable on tax unit (e.g. eink_st_tu) on hh level (e.g. arbeitsl_geld_2_eink_m_hh).
This is currently implemented by calculating an individual variable (eink_st_tu / anz_erwachsen_tu) and aggregating this variable on hh level.
Problem is that the individual variable does not have a clear interpretation (see the related issue #445).
Also it seems to be not the most efficient solution.

Describe the solution you would like to see

Add new aggregation functions which e.g. calculate eink_st_hh based on eink_st_tu
Aggregation functions currently take an individual-level variable and a grouping id as inputs (e.g. def grouped_sum(column, group_id):)
New functions should take a group-level variable and two grouping ids as inputs (e.g. def grouped_sum_from_grouped(column, group_id_input, group_id_output):)

Describe alternatives you've considered

Leave as is.

hmgaudecker commented 1 year ago

Let's see after #509 is implemented how many nested groupings are left (currently, tu always is a subset of hh, but not many things like that might be left).

MImmesberger commented 5 months ago

Given that we don't want to implement #701 we can close this as well, right?

hmgaudecker commented 5 months ago

Probably yes. Is #445, which triggered this, still relevant? And when can we be sure that one group is a subset of another? Would we need a check?

MImmesberger commented 5 months ago

Correct me if I'm wrong:

The following groups should not be subsets of each other:

sn and hh or fg or bg (taxes can be filed jointly even when getrennt lebend)
ehe and bg or hh or eg (getrennt lebend)

The following should always be subsets:

eg is a subset of bg
bg is always a subset of fg
fg is always a subset of hh

We could make aggregations possible for the second list and rely on user-provided aggregations for the first list.

hmgaudecker commented 5 months ago

Cool, then let's keep this open for the moment. We will need to worry about how to ensure nesting then. Maybe we should restrict ourselves to sums, too.