[BUG-REPORT] `vaex.agg.list` combined with `col.str.join(',')` is corrupting the dataframe

Thank you for reaching out and helping us improve Vaex!

Before you submit a new Issue, please read through the documentation. Also, make sure you search through the Open and Closed Issues - your problem may already be discussed or addressed.

Description Please provide a clear and concise description of the problem. This should contain all the steps needed to reproduce the problem. A minimal code example that exposes the problem is very appreciated.

Software information

Vaex version (import vaex; vaex.__version__): 4.9.1 & 4.10
Vaex was installed via: pip / conda-forge / from source pip
OS: Linux + macOS

import numpy as np
import vaex

text = vaex.from_arrays(
    id=list(range(10_000)),
    text=[f"sentence number {i}" for i in range(10_000)],
)
df = vaex.from_arrays(
    id=list(range(25_000)),
    sample_id=[i%10_000 for i in range(25_000)],
    score=np.random.rand(25_000)
)

text.export("text_data.hdf5")
df.export("data.hdf5")

text_df = vaex.open("text_data.hdf5")
data_df = vaex.open("data.hdf5")

df = data_df.join(text_df, left_on="sample_id", right_on="id", rsuffix="_R").drop("id_R")

df["result"] = '{"input":' + '"' + df.text + '"' + ', "score":' + df.score.astype('str') + "}"
df = df.groupby("sample_id", agg={"result": vaex.agg.list})  # THIS IS THE LINE THAT BREAKS THINGS

df["result"] = "[" + df.result.str.join(",") + "]"
display(df[df["sample_id"]==8668].result.tolist())

df.export("test.csv") 

newdf = vaex.open('test.csv')
display(newdf[newdf["sample_id"]==8668].result.tolist())

If you swap that broken line commented, and use a custom function for the desired results

import pyarrow as pa
@vaex.register_function()
def format_json(arr):
    return [",".join(a.as_py()) for a in arr]

and call df["result"] = "[" + df.result.format_json() + "]" instead, everything works

vaexio / vaex

[BUG-REPORT] `vaex.agg.list` combined with `col.str.join(',')` is corrupting the dataframe #2111