inplace warning ahead of pandas 3.0

[jefferis]

I'm seeing this (a lot) with my new caveclient install.

/Users/jefferis/Library/r-miniconda-arm64/envs/r-reticulate/lib/python3.10/site-packages/caveclient/materializationengine.py:1085: FutureWarning: A value is trying to be set on a copy of a DataFrame or Series through chained assignment using an inplace method.
The behavior will change in pandas 3.0. This inplace method will never work because the intermediate object on which we are setting values always behaves as a copy.

For example, when doing 'df[col].method(value, inplace=True)', try using 'df.method({col: value}, inplace=True)' or df[col] = df[col].method(value) instead, to perform the operation inplace on the original object.

https://github.com/CAVEconnectome/CAVEclient/blame/863eeb21879729c8ebc5e4c64ce742e6906733e4/caveclient/materializationengine.py#L1085

pandas is at version 2.2.2 and this is a summary of the rest of my Python env.

Python
----
python:         /Users/jefferis/Library/r-miniconda-arm64/envs/r-reticulate/bin/python
libpython:      /Users/jefferis/Library/r-miniconda-arm64/envs/r-reticulate/lib/libpython3.10.dylib
pythonhome:     /Users/jefferis/Library/r-miniconda-arm64/envs/r-reticulate:/Users/jefferis/Library/r-miniconda-arm64/envs/r-reticulate
version:        3.10.14 | packaged by conda-forge | (main, Mar 20 2024, 12:51:49) [Clang 16.0.6 ]
numpy:          /Users/jefferis/Library/r-miniconda-arm64/envs/r-reticulate/lib/python3.10/site-packages/numpy
numpy_version:  1.26.4

NOTE: Python version was forced by use_python() function

          package version         requirement channel installed  core
1    cloud-volume  8.33.0 cloud-volume=8.33.0    pypi      TRUE  TRUE
2      caveclient  5.21.0   caveclient=5.21.0    pypi      TRUE  TRUE
3    seatable-api   2.8.0  seatable-api=2.8.0    pypi      TRUE  TRUE
4         pyarrow  16.1.0      pyarrow=16.1.0    pypi      TRUE FALSE
5         fafbseg   3.0.8       fafbseg=3.0.8    pypi      TRUE FALSE
6         DracoPy    <NA>                <NA>    <NA>     FALSE FALSE
7       meshparty    <NA>                <NA>    <NA>     FALSE FALSE
8        skeletor   1.3.0      skeletor=1.3.0    pypi      TRUE FALSE
9        pykdtree    <NA>                <NA>    <NA>     FALSE FALSE
10       pyembree    <NA>                <NA>    <NA>     FALSE FALSE
11 pychunkedgraph    <NA>                <NA>    <NA>     FALSE FALSE
12        igneous    <NA>                <NA>    <NA>     FALSE FALSE
13      fastremap  1.14.1    fastremap=1.14.1    pypi      TRUE FALSE
14      ncollpyde  0.19.0    ncollpyde=0.19.0    pypi      TRUE FALSE

[jefferis]

Python example:

>>> import caveclient
>>> import pandas as pd
>>> import datetime
>>> datastack_name = "flywire_fafb_public"
>>> client = caveclient.CAVEclient(datastack_name)
>>> postsyn_df = client.materialize.live_query("synapses_nt_v1", filter_in_dict={"post_pt_root_id": [720575940617343316]}, timestamp=datetime.datetime.now())
/Users/jefferis/Library/r-miniconda-arm64/envs/r-reticulate/lib/python3.10/site-packages/caveclient/materializationengine.py:1085: ChainedAssignmentError: A value is trying to be set on a copy of a DataFrame or Series through chained assignment using an inplace method.
When using the Copy-on-Write mode, such inplace method never works to update the original DataFrame or Series, because the intermediate object on which we are setting values always behaves as a copy.

For example, when doing 'df[col].method(value, inplace=True)', try using 'df.method({col: value}, inplace=True)' instead, to perform the operation inplace on the original object.

  df[root_id_col].replace(future_map, inplace=True)

R example

library(fafbseg)
flywire_partner_summary('720575940617343316', method = 'cave')

[fcollman]

thanks for this greg.. on our radar now. we should proactively pin pandas<3.0 for now till we get this warning taking care of.

[jefferis]

Thanks @fcollman! I'm getting this with 2.2.2 so I think the pin would have to be a bit more aggressive than <3.0 and might cause trouble elsewhere. I guess the warnings are designed to be annoying enough to make people bite the bullet and fix!

[fcollman]

this is just a warning though right at 2.2.2 ?

[fcollman]

I've pinned it below 3.0 for now

[jefferis]

It's just a warning at 2.2.2 but it's a very noisy one!

[bdpedigo]

fixed in ed0b1aecf3a2083be1b8bc6a97bd9c97ab42b4ee (in addition to the pin)