quinnj / JSON3.jl

Other
214 stars 47 forks source link

DataFrameRows iterator (from DataFrames.eachrow) repeats entire dataframe when serialising with JSON3 #280

Open ahjulstad opened 5 months ago

ahjulstad commented 5 months ago

Example:

using JSON3, DataFrames

rc = 1000
df = DataFrame(a=1:rc, b=rand('a':'z', rc), c=rand(rc))

println(length(JSON3.write(df)))
println(length(JSON3.write(eachrow(df))))

prints

27327   
27426787

I stumbled upon this trying a naïve approach for outputing the dataframe in record mode. The easiest workaround for me was to use TypedTables and just make a Table out if the dataframe first (instead of creating an iterator).