Compound keys for splitby

Shine226 commented 3 years ago

Wiggum app is not support for two or more variable as compounding splitby.

Shine226 commented 3 years ago

It is related to one of the MODIN issue #202.

Shine226 commented 3 years ago

For example, use ['gender', 'race'] as splitby

brownsarahm commented 3 years ago

What are you recommending as a change here?

Shine226 commented 3 years ago

Modin doesn't support keys:

import modin.pandas as pd
groupby_name_by_type = {pd.groupby.DataFrameGroupBy:lambda df: df.keys,
                                pd.dataframe.DataFrame:lambda df: None}

new error:

Traceback (most recent call last):
  File "/opt/anaconda3/lib/python3.7/site-packages/flask/app.py", line 2463, in __call__
    return self.wsgi_app(environ, start_response)
  File "/opt/anaconda3/lib/python3.7/site-packages/flask/app.py", line 2449, in wsgi_app
    response = self.handle_exception(e)
  File "/opt/anaconda3/lib/python3.7/site-packages/flask/app.py", line 1866, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/opt/anaconda3/lib/python3.7/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/opt/anaconda3/lib/python3.7/site-packages/flask/app.py", line 2446, in wsgi_app
    response = self.full_dispatch_request()
  File "/opt/anaconda3/lib/python3.7/site-packages/flask/app.py", line 1951, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/opt/anaconda3/lib/python3.7/site-packages/flask/app.py", line 1820, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/opt/anaconda3/lib/python3.7/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/opt/anaconda3/lib/python3.7/site-packages/flask/app.py", line 1949, in full_dispatch_request
    rv = self.dispatch_request()
  File "/opt/anaconda3/lib/python3.7/site-packages/flask/app.py", line 1935, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/Users/chenguangxu/Documents/GitHub/detect_simpsons_paradox_dev/wiggum_app/controller.py", line 274, in main
    labeled_df_setup.get_subgroup_trends_1lev(trend_list)
  File "/Users/chenguangxu/Documents/GitHub/detect_simpsons_paradox_dev/wiggum/detectors.py", line 284, in get_subgroup_trends_1lev
    curgroup_trend_df = cur_trend.get_trends(cur_grouping,'subgroup_trend')
  File "/Users/chenguangxu/Documents/GitHub/detect_simpsons_paradox_dev/wiggum/trend_components/statistical.py", line 173, in get_trends
    groupby_name = groupby_name_by_type[type(data_df)](data_df)
  File "/Users/chenguangxu/Documents/GitHub/detect_simpsons_paradox_dev/wiggum/trend_components/statistical.py", line 6, in <lambda>
    groupby_name_by_type = {pd.groupby.DataFrameGroupBy:lambda df: df.keys,
  File "/opt/anaconda3/lib/python3.7/site-packages/modin/pandas/groupby.py", line 125, in __getattr__
    raise e
  File "/opt/anaconda3/lib/python3.7/site-packages/modin/pandas/groupby.py", line 121, in __getattr__
    return object.__getattribute__(self, key)
AttributeError: 'DataFrameGroupBy' object has no attribute 'keys'

fairnessforensics / wiggum

Compound keys for splitby #204