BUG: Series attrs disappear after modifcation to the DataFrame that contains them.

[lendres]

Pandas version checks

• [X] I have checked that this issue has not already been reported.

• [X] I have confirmed this bug exists on the latest version of pandas.

• [ ] I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import pandas as pd

# Example 1 - changing the data type.
df = pd.DataFrame(dict(column1=[1,2,3,4], column2=[1,2,3,4]))
df["column1"].attrs['name'] = 'hello'

print("\nExample 1 before:")
print(df)
print("Column1 attributes:", df["column1"].attrs)

# Changing the data type of the DataFrame drops the series attributes.
df1 = df.astype(float)
print("\nAfter type change:")
print(df1)
print("Column1 attributes:", df1["column1"].attrs)

# Example 2 - assignment to a DataFrame.
df = pd.DataFrame(dict(column1=[1,2,3,4], column2=[1,2,3,4]))
df["column1"].attrs['name'] = 'hello1'
df["column2"].attrs['name'] = 'hello2'

print("\n\n\nExample 2 before:")
print(df)
print("Column1 attributes:", df["column1"].attrs)
print("Column2 attributes:", df["column2"].attrs)

# Assigning a column to the DataFrame drops the attributes from the Series in DataFrame.
df["column1"] = df["column1"] + 5
print("\nAfter assignment to DataFrame:")
print(df)
print("Column1 attributes:", df["column1"].attrs)
print("Column2 attributes:", df["column2"].attrs)

Issue Description

Issue

The attrs dictionary of Series are not carried through when operations are performed on a DataFrame.

Example 1

# Example 1 - changing the data type.
df = pd.DataFrame(dict(column1=[1,2,3,4], column2=[1,2,3,4]))
df["column1"].attrs['name'] = 'hello'

print("\nExample 1 before:")
print(df)
print("Column1 attributes:", df["column1"].attrs)

"""
OUTUPUT:
Example 1 before:
   column1  column2
0        1        1
1        2        2
2        3        3
3        4        4
Column1 attributes: {'name': 'hello'}
"""

The first series has an attr stored. However, after changing the data types, the attribute is lost.

# Changing the data type of the DataFrame drops the series attributes.
df1 = df.astype(float)
print("\nAfter type change:")
print(df1)
print("Column1 attributes:", df1["column1"].attrs)

"""
OUTUPUT:
After type change:
   column1  column2
0      1.0      1.0
1      2.0      2.0
2      3.0      3.0
3      4.0      4.0
Column1 attributes: {}         <-- NO ATTRS
"""

Example 2

# Example 2 - assignment to a DataFrame.
df = pd.DataFrame(dict(column1=[1,2,3,4], column2=[1,2,3,4]))
df["column1"].attrs['name'] = 'hello1'
df["column2"].attrs['name'] = 'hello2'

print("\n\n\nExample 2 before:")
print(df)
print("Column1 attributes:", df["column1"].attrs)
print("Column2 attributes:", df["column2"].attrs)

"""
OUTPUT:
Example 2 before:
   column1  column2
0        1        1
1        2        2
2        3        3
3        4        4
Column1 attributes: {'name': 'hello1'}
Column2 attributes: {'name': 'hello2'}
"""

The first series has an attr stored. However, reassigning a column drops the attributes.

# Assigning a column to the DataFrame drops the attributes from the Series in DataFrame.
df["column1"] = df["column1"] + 5
print("\nAfter assignment to DataFrame:")
print(df)
print("Column1 attributes:", df["column1"].attrs)
print("Column2 attributes:", df["column2"].attrs)

"""
OUTPUT:
After assignment to DataFrame:
   column1  column2
0        6        1
1        7        2
2        8        3
3        9        4
Column1 attributes: {}        <-- NO ATTRS
Column2 attributes: {}        <-- NO ATTRS
"""

Expected Behavior

The attributes (attrs) should be maintained in the DataFrame when operations are performed on it.

Tested on 2.1.1 and 2.2.0

Installed Versions

INSTALLED VERSIONS ------------------ commit : f538741432edf55c6b9fb5d0d496d2dd1d7c2457 python : 3.11.4.final.0 python-bits : 64 OS : Windows OS-release : 10 Version : 10.0.19044 machine : AMD64 processor : Intel64 Family 6 Model 165 Stepping 2, GenuineIntel byteorder : little LC_ALL : None LANG : en LOCALE : English_United States.1252 pandas : 2.2.0 numpy : 1.26.3 pytz : 2024.1 dateutil : 2.8.2 setuptools : 68.0.0 pip : 23.2.1 Cython : None pytest : None hypothesis : None sphinx : None blosc : None feather : None xlsxwriter : None lxml.etree : 4.9.3 html5lib : None pymysql : None psycopg2 : None jinja2 : 3.1.2 IPython : 8.15.0 pandas_datareader : None adbc-driver-postgresql: None adbc-driver-sqlite : None bs4 : 4.12.2 bottleneck : None dataframe-api-compat : None fastparquet : None fsspec : None gcsfs : None matplotlib : 3.7.1 numba : None numexpr : 2.8.4 odfpy : None openpyxl : 3.1.2 pandas_gbq : None pyarrow : None pyreadstat : None python-calamine : None pyxlsb : None s3fs : None scipy : 1.11.1 sqlalchemy : None tables : None tabulate : None xarray : None xlrd : None zstandard : None tzdata : 2023.4 qtpy : None pyqt5 : None

[rhshadrach]

Thanks for the report; attr propagation is only partially implemented currently. PRs to fix are welcome!

[dontgoto]

take

[dontgoto]

On main this type of Series.attr propagation is basically nonexistent. Practically, only DataFrame.attrs are accessible and get attached to the Series when the Series representation of a column is retrieved.

I would try to implement a solution that introduces hidden column_attrs to the DataFrame that would support this type of Series.attrs propagation. Solutions that only make use of the DataFrame.attrs by design either lack propagation functionality or lead to confusing behaviour.