pyarrow ValueError: Duplicate column names found

[citynorman]

pyarrow can't save duplicate columns

[d6tdev]

To find out which column:

df.columns[df.columns.duplicated()]
# or
from collections import Counter
Counter(df.columns).most_common()[:5]

To fix it:

• this doesn't work df.rename(columns={'name':'name2'})
• make sure you are concat/joining correctly
• https://stackoverflow.com/questions/24685012/pandas-dataframe-renaming-multiple-identically-named-columns