Groupby with more than two columns isn't possible

[dom-devel]

Describe the bug I'm a little unsure if this is a bug? Or some sort of un-documented restriction?

If you try to groupby with 3 columns, it only groups by the first. If you try to groupby with 2 columns, it only groups by both.

Version

• Browser version of danfo.
• danfo - 0.2.5

To Reproduce

• Generate a dataframe with 3 text columns and one numeric column.
• Run a groupby - const dfGroup = df.groupby(["col1", "col2","col3"])
• Perform an aggregation dfGroup.col(["numeric_col"]).sum()

Expected behavior I get a dataframe with aggregation by all 3 columns. Instead I'm getting a dataframe with aggregation by a single column.

Is this currently a restriction or is this a bug?

[steveoni]

that's true, currently it only group by two column

[risenW]

@steveoni Did you fix this in the recent PR, or should we move this to future plans?

[steveoni]

@risenW not fixed, it will require some refactoring to the groupby class, we can add it to the future plans

[sponsfreixes]

I had a look at it, and I think I can have a patch in a few days.

[steveoni]

@sponsfreixes that will be great.

I plan of refactoring and structuring the groupby operation properly. but time factor.

you can go ahead and add your changes

[sponsfreixes]

I started working on it this last weekend, but forgot to post here 😅. It's a pretty fun refactoring because to support any number of columns you need to use recursion to transverse nested dictionaries as trees. I'll make a PR once I'm done.

[steveoni]

@sponsfreixes That's cool. I will be expecting your PR 👍

[sponsfreixes]

While working on it I think I discovered a bug (or either I'm misunderstanding something). I am getting duplicated rows on the following example:

const dfd = require("danfojs-node");
let data = { 'A': [ 'foo', 'bar', 'foo', 'bar',
      'foo', 'bar', 'foo', 'foo' ],
    'B': [ 'one', 'one', 'two', 'three',
      'two', 'two', 'one', 'three' ],
    'C': [ 1, 3, 2, 4, 5, 2, 6, 7 ],
    'D': [ 3, 2, 4, 1, 5, 6, 7, 8 ] };
let df = new dfd.DataFrame(data);
let group_df = df.groupby([ "A" ]);
group_df.col(['C', 'D']).apply((x) => x.add(2)).print()

I get back:

╔════╤═══════════════════╤═══════════════════╤═══════════════════╗
║    │ A                 │ C_apply           │ D_apply           ║
╟────┼───────────────────┼───────────────────┼───────────────────╢
║ 0  │ foo               │ 3                 │ 5                 ║
╟────┼───────────────────┼───────────────────┼───────────────────╢
║ 1  │ foo               │ 4                 │ 6                 ║
╟────┼───────────────────┼───────────────────┼───────────────────╢
║ 2  │ foo               │ 7                 │ 7                 ║
╟────┼───────────────────┼───────────────────┼───────────────────╢
║ 3  │ foo               │ 8                 │ 9                 ║
╟────┼───────────────────┼───────────────────┼───────────────────╢
║ 4  │ foo               │ 9                 │ 10                ║
╟────┼───────────────────┼───────────────────┼───────────────────╢
║ 5  │ foo               │ 3                 │ 5                 ║
╟────┼───────────────────┼───────────────────┼───────────────────╢
║ 6  │ foo               │ 4                 │ 6                 ║
╟────┼───────────────────┼───────────────────┼───────────────────╢
║ 7  │ foo               │ 7                 │ 7                 ║
╟────┼───────────────────┼───────────────────┼───────────────────╢
║ 8  │ foo               │ 8                 │ 9                 ║
╟────┼───────────────────┼───────────────────┼───────────────────╢
║ 9  │ foo               │ 9                 │ 10                ║
╟────┼───────────────────┼───────────────────┼───────────────────╢
║ 10 │ bar               │ 5                 │ 4                 ║
╟────┼───────────────────┼───────────────────┼───────────────────╢
║ 11 │ bar               │ 6                 │ 3                 ║
╟────┼───────────────────┼───────────────────┼───────────────────╢
║ 12 │ bar               │ 4                 │ 8                 ║
╟────┼───────────────────┼───────────────────┼───────────────────╢
║ 13 │ bar               │ 5                 │ 4                 ║
╟────┼───────────────────┼───────────────────┼───────────────────╢
║ 14 │ bar               │ 6                 │ 3                 ║
╟────┼───────────────────┼───────────────────┼───────────────────╢
║ 15 │ bar               │ 4                 │ 8                 ║
╚════╧═══════════════════╧═══════════════════╧═══════════════════╝

Row 0 is duplicated with row 5, row 1 with row 6, etc. Then for the "bar" group, the same happens with 10-13, 11-14, etc. Is that really the expected behavior?

[steveoni]

@sponsfreixes that's true, I've fixed that in edd2e0c34d7002d8a38dd987b20f998bd95572b6 and its already merged. so pull the new update or check the commit and make adjustment to the version you have

[steveoni]

@dom-devel you can now group by more than two columns in the new update