Converting grouped to DataFrame

[geyang]

currently there is not a good way to convert a grouped DataFrame to a DataFrame.

The GroupedDataFrame class is also missing toDict method.

[Gmousse]

Hi @episodeyang,

Can you give me a usage example about converting a grouped DataFrame into a DataFrame ?

It could be interesting in order to have a DataFrame having multiple DataFrames in each row, but I don't know if it's not too tricky and really useful.

[geyang]

Here I have a data frame with keys:

["id", "metricFiles", "Args.start_seed", "Args.lr", "Args.weight_decay"]

I want to average those which have the same learning rate and weight decay parameters.

Because the grouped data frame does not have toDict methods, I need to declare a new one.

  const grouped = sorted.groupBy(...groupKeys)
      .aggregate(group => group.select(...intersect(aggKeys, allKeys)).toDict());

  df = new DataFrame({
    ...grouped.select(...groupKeys).toDict(),
    ...new DataFrame(grouped.select('aggregation').toDict().aggregation || []).toDict()
  });

  const expList = df.toCollection();
  console.log(expList);

[Gmousse]

Hi @episodeyang,

In understand the need. I will see to implement a solution.

[Gmousse]

Closed due to inactivity. We will repoen it if necessary.