Closed mwouts closed 1 year ago
This is probably because the memory used by the table is computed using df.values.nbytes, which empirically seems to take a much longer time than sum(df[col].values.nbytes for col in df.columns).
df.values.nbytes
sum(df[col].values.nbytes for col in df.columns)
The new version is significantly faster: 3.45 milliseconds vs 1.38 seconds
This is probably because the memory used by the table is computed using
df.values.nbytes
, which empirically seems to take a much longer time thansum(df[col].values.nbytes for col in df.columns)
.