vaexio / vaex

Out-of-Core hybrid Apache Arrow/NumPy DataFrame for Python, ML, visualization and exploration of big tabular data at a billion rows per second 🚀
https://vaex.io
MIT License
8.27k stars 590 forks source link

[BUG-REPORT] Rename twice then concat makes error #2297

Open bls-lehoai opened 1 year ago

bls-lehoai commented 1 year ago

Description I have 2 tables as below: image Column name: Field0, Field1, Field2 image Column name: Field0, Field1, Field2, Field3

for some logic app reason, I rename the table 2 columns to Field3, Field4, Field5, and Field6, but Field3 already exists so i can't rename Field0 -> Field3, so first i rename Field0 to 'hField3h' (attach 'h 'at prefix and suffix), do the same thing for the rest, finally then rename again all column by removing 'h' character ('hField3h' => 'Field3')

after that, i do 'concat' 2 tables and get an error when print result data-frame

image

The code is bellow u can test it by yourself

import vaex

df1 = vaex.open("C:\\Users\\Admin\\Documents\\hoai13\\hoai1.hdf5")
df2 = vaex.open("C:\\Users\\Admin\\Documents\\hoai13\\hoai2.hdf5")
print(df1)
print(df2)
df2.rename("Field0", "hField3h")
df2.rename("Field1", "hField4h")
df2.rename("Field2", "hField5h")
df2.rename("Field3", "hField6h")
df2.rename("hField3h", "Field3")
df2.rename("hField4h", "Field4")
df2.rename("hField5h", "Field5")
df2.rename("hField6h", "Field6")
print(df2)
df3 = vaex.concat([df2, df1])

print(df3)

HDF5 files:

data.zip

bls-lehoai commented 1 year ago

No update? :(