IndexError: index (1945227) out of range

[nynkekatinka]

Hi @ckmah!

I get the same ValueError as raised in the other current issue. In addition, when running bt.tl.lp(adata) in the example pipeline, I get the following error:

When you run bt.tl.lp(adata, copy=True) it does run succesfully, but given that you made a copy of your data, and cannot store that in a local variable, you can't really continue in the pipeline.

Would love to be updated whenever the bugs are fixed!

[ckmah]

Hi @nynkekatinka , for a temporary fix, you can reload the sample data before running bt.tl.lp() as we do here. https://github.com/ckmah/bento-tools/pull/101

We'll take a look!

[dylanclam12]

Hi @nynkekatinka do you mind sharing the the list of dependencies and their versions so I can take a look at this error? Seems like I'm not running into the same thing when I run bento.tl.lp.

[nynkekatinka]

Sure thing @dylanclam12, see the list with all the packages and their versions below. Please let me know if you'd need anything else.

packages in environment at /nynke/miniconda3/envs/bento:

Name Version Build Channel _libgcc_mutex 0.1 main _openmp_mutex 5.1 1_gnu adjusttext 0.7.3 pypi_0 pypi affine 2.4.0 pypi_0 pypi anndata 0.8.0 pypi_0 pypi astropy 5.2.2 pypi_0 pypi asttokens 2.4.0 pypi_0 pypi attrs 23.1.0 pypi_0 pypi backcall 0.2.0 pyhd3eb1b0_0 anaconda bento-tools 2.0.0 pypi_0 pypi bzip2 1.0.8 h7f98852_4 conda-forge c-ares 1.19.1 h5eee18b_0 ca-certificates 2023.08.22 h06a4308_0 anaconda certifi 2023.7.22 pypi_0 pypi click 8.1.7 pypi_0 pypi click-plugins 1.1.1 pypi_0 pypi cligj 0.7.2 pypi_0 pypi cmake 3.26.4 h96355d8_0 comm 0.1.4 pypi_0 pypi contourpy 1.1.1 pypi_0 pypi cycler 0.12.0 pypi_0 pypi debugpy 1.8.0 pypi_0 pypi decorator 5.1.1 pyhd3eb1b0_0 anaconda decoupler 1.5.0 pypi_0 pypi emoji 1.7.0 pypi_0 pypi executing 2.0.0 pypi_0 pypi expat 2.5.0 h6a678d5_0 fiona 1.9.4.post1 pypi_0 pypi fonttools 4.43.0 pypi_0 pypi gdal 1.9.2 0 bioconda geopandas 0.10.2 pypi_0 pypi h5py 3.9.0 pypi_0 pypi importlib-metadata 6.8.0 pypi_0 pypi importlib-resources 6.1.0 pypi_0 pypi importlib_metadata 6.0.0 hd3eb1b0_0 anaconda ipykernel 6.25.2 pypi_0 pypi ipython 8.12.3 pypi_0 pypi ipywidgets 8.1.1 pypi_0 pypi jedi 0.19.1 pypi_0 pypi joblib 1.3.2 pypi_0 pypi jupyter-client 8.3.1 pypi_0 pypi jupyter-core 5.3.2 pypi_0 pypi jupyter_client 8.1.0 py38h06a4308_0 anaconda jupyter_core 5.3.0 py38h06a4308_0 anaconda jupyterlab-widgets 3.0.9 pypi_0 pypi kiwisolver 1.4.5 pypi_0 pypi kneed 0.8.5 pypi_0 pypi krb5 1.20.1 h143b758_1 ld_impl_linux-64 2.38 h1181459_1 libcurl 8.2.1 h251f7ec_0 libedit 3.1.20221030 h5eee18b_0 libev 4.33 h516909a_1 conda-forge libffi 3.4.4 h6a678d5_0 libgcc 7.2.0 h69d50b8_2 conda-forge libgcc-ng 11.2.0 h1234567_1 libgomp 11.2.0 h1234567_1 libnghttp2 1.52.0 h2d74bed_1 libsodium 1.0.18 h7b6447c_0 anaconda libssh2 1.10.0 hdbd6064_2 libstdcxx-ng 11.2.0 h1234567_1 libuv 1.44.2 h5eee18b_0 llvmlite 0.41.0 pypi_0 pypi lz4-c 1.9.4 h6a678d5_0 matplotlib 3.7.3 pypi_0 pypi matplotlib-inline 0.1.6 py38h06a4308_0 anaconda matplotlib-scalebar 0.8.1 pypi_0 pypi minisom 2.3.1 pypi_0 pypi natsort 8.4.0 pypi_0 pypi ncurses 6.4 h6a678d5_0 nest-asyncio 1.5.8 pypi_0 pypi networkx 3.1 pypi_0 pypi nose 1.3.7 pypi_0 pypi numba 0.58.0 pypi_0 pypi numpy 1.24.4 pypi_0 pypi openssl 3.0.11 h7f8727e_2 anaconda packaging 23.2 pypi_0 pypi pandas 1.5.3 pypi_0 pypi pandoc 2.19.2 ha770c72_0 conda-forge parso 0.8.3 pyhd3eb1b0_0 anaconda patsy 0.5.3 pypi_0 pypi pexpect 4.8.0 pyhd3eb1b0_3 anaconda pickleshare 0.7.5 pyhd3eb1b0_1003 anaconda pillow 10.0.1 pypi_0 pypi pip 23.2.1 py38h06a4308_0 platformdirs 3.11.0 pypi_0 pypi prompt-toolkit 3.0.39 pypi_0 pypi psutil 5.9.5 pypi_0 pypi ptyprocess 0.7.0 pyhd3eb1b0_2 anaconda pure_eval 0.2.2 pyhd3eb1b0_0 anaconda pyerfa 2.0.0.3 pypi_0 pypi pygeos 0.12.0 pypi_0 pypi pygments 2.16.1 pypi_0 pypi pynndescent 0.5.10 pypi_0 pypi pyparsing 3.1.1 pypi_0 pypi pyproj 3.5.0 pypi_0 pypi python 3.8.18 h955ad1f_0 python-dateutil 2.8.2 pyhd3eb1b0_0 anaconda pytz 2023.3.post1 pypi_0 pypi pyyaml 6.0.1 pypi_0 pypi pyzmq 25.1.1 pypi_0 pypi rasterio 1.3.8 pypi_0 pypi readline 8.2 h5eee18b_0 rhash 1.4.3 hdbd6064_0 scanpy 1.9.5 pypi_0 pypi scikit-learn 1.3.1 pypi_0 pypi scipy 1.10.1 pypi_0 pypi seaborn 0.12.2 pypi_0 pypi session-info 1.0.0 pypi_0 pypi setuptools 68.0.0 py38h06a4308_0 shapely 1.8.5.post1 pypi_0 pypi six 1.16.0 pyhd3eb1b0_1 anaconda snuggs 1.4.7 pypi_0 pypi sparse 0.13.0 pypi_0 pypi sqlite 3.41.2 h5eee18b_0 stack-data 0.6.3 pypi_0 pypi stack_data 0.2.0 pyhd3eb1b0_0 anaconda statsmodels 0.13.5 pypi_0 pypi stdlib-list 0.9.0 pypi_0 pypi tbb 2021.10.0 pypi_0 pypi tensorly 0.7.0 pypi_0 pypi threadpoolctl 3.2.0 pypi_0 pypi tk 8.6.12 h1ccaba5_0 tornado 6.3.3 py38h5eee18b_0 anaconda tqdm 4.66.1 pypi_0 pypi traitlets 5.11.2 pypi_0 pypi typing-extensions 4.8.0 pypi_0 pypi typing_extensions 4.7.1 py38h06a4308_0 anaconda umap-learn 0.5.4 pypi_0 pypi upsetplot 0.6.1 pypi_0 pypi wcwidth 0.2.8 pypi_0 pypi wheel 0.41.2 py38h06a4308_0 widgetsnbextension 4.0.9 pypi_0 pypi xgboost 1.4.0 pypi_0 pypi xz 5.4.2 h5eee18b_0 zeromq 4.3.4 h2531618_0 anaconda zipp 3.17.0 pypi_0 pypi zlib 1.2.13 h5eee18b_0 zstd 1.5.5 hc292b87_0

[nynkekatinka]

@dylanclam12 @ckmah when I run this on my own data, I run into the same error, whether I make a shallow copy or not. The temporary fix of reloading the data does indeed work, however I am now running into a new error. I included a picture of the truncated error, and copied the text of the complete error.

Looking forward to hear your thoughts.

{ "name": "ValueError", "message": "max() arg is an empty sequence", "stack": "--------------------------------------------------------------------------- ValueError Traceback (most recent call last) /media/gambino/students_workdir/nynke/comparative_tools/bento.ipynb Cell 24 line 1 ----> 1 bt.tl.lp(adata)

File /media/gambino/students_workdir/nynke/miniconda3/envs/bento/lib/python3.8/site-packages/bento/_utils.py:51, in track..wrapper(*args, *kwds) 49 new_attr = list_attributes(out_adata) 50 else: ---> 51 func(args, **kwds) 52 new_attr = list_attributes(adata) 54 # Print differences between new and old adata

File /media/gambino/students_workdir/nynke/miniconda3/envs/bento/lib/python3.8/site-packages/bento/tools/_lp.py:49, in lp(data, groupby, copy) 45 featurekey = f\"cell{'_'.join(groupby)}_features\" 46 if feature_key not in adata.uns.keys() or not all( 47 f in adata.uns[feature_key].columns for f in PATTERN_FEATURES 48 ): ---> 49 bento.tl.analyze_points( 50 adata, 51 \"cell_shape\", 52 [\"proximity\", \"asymmetry\", \"ripley\", \"point_dispersion_norm\"], 53 groupby=groupby, 54 recompute=True, 55 ) 56 bento.tl.analyze_points( 57 adata, 58 \"nucleus_shape\", (...) 61 recompute=True, 62 ) 63 X_df = adata.uns[feature_key][PATTERN_FEATURES]

File /media/gambino/students_workdir/nynke/miniconda3/envs/bento/lib/python3.8/site-packages/bento/_utils.py:51, in track..wrapper(*args, *kwds) 49 new_attr = list_attributes(out_adata) 50 else: ---> 51 func(args, **kwds) 52 new_attr = list_attributes(adata) 54 # Print differences between new and old adata

File /media/gambino/students_workdir/nynke/miniconda3/envs/bento/lib/python3.8/site-packages/bento/tools/_point_features.py:153, in analyze_points(data, shape_names, feature_names, groupby, recompute, progress, copy) 151 for start, end in group_locs: 152 cell_points = points_df.iloc[start:end] --> 153 output.append(process_partition(cell_points)) 154 output = pd.concat(output) 156 # Save and overwrite existing

File /media/gambino/students_workdir/nynke/miniconda3/envs/bento/lib/python3.8/site-packages/bento/tools/_point_features.py:134, in analyze_points..process_partition(partition_df) 132 def process_partition(partition_df): 133 # Groupby by cell and groupby keys and process each sample --> 134 out = partition_df.groupby(groupby, observed=True).apply(process_sample) 135 return pd.DataFrame.from_records(out.values, index=out.index)

File /media/gambino/students_workdir/nynke/miniconda3/envs/bento/lib/python3.8/site-packages/pandas/core/groupby/groupby.py:1567, in GroupBy.apply(self, func, *args, *kwargs) 1559 new_msg = ( 1560 f\"The operation {orig_func} failed on a column. If any error is \" 1561 f\"raised, this will raise an exception in a future version \" 1562 f\"of pandas. Drop these columns to avoid this warning.\" 1563 ) 1564 with rewrite_warning( 1565 old_msg, FutureWarning, new_msg 1566 ) if is_np_func else nullcontext(): -> 1567 result = self._python_apply_general(f, self._selected_obj) 1568 except TypeError: 1569 # gh-20949 1570 # try again, with .apply acting as a filtering (...) 1574 # fails on some* columns, e.g. a numeric operation 1575 # on a string grouper column 1577 with self._group_selection_context(): 1578 # GH#50538

File /media/gambino/students_workdir/nynke/miniconda3/envs/bento/lib/python3.8/site-packages/pandas/core/groupby/groupby.py:1629, in GroupBy._python_apply_general(self, f, data, not_indexed_same, is_transform, is_agg) 1592 @final 1593 def _python_apply_general( 1594 self, (...) 1599 is_agg: bool = False, 1600 ) -> NDFrameT: 1601 \"\"\" 1602 Apply function f in python space 1603 (...) 1627 data after applying f 1628 \"\"\" -> 1629 values, mutated = self.grouper.apply(f, data, self.axis) 1630 if not_indexed_same is None: 1631 not_indexed_same = mutated or self.mutated

File /media/gambino/students_workdir/nynke/miniconda3/envs/bento/lib/python3.8/site-packages/pandas/core/groupby/ops.py:839, in BaseGrouper.apply(self, f, data, axis) 837 # group might be modified 838 group_axes = group.axes --> 839 res = f(group) 840 if not mutated and not _is_indexed_like(res, group_axes, axis): 841 mutated = True

File /media/gambino/students_workdir/nynke/miniconda3/envs/bento/lib/python3.8/site-packages/bento/tools/_point_features.py:128, in analyze_points..process_sample(df) 126 sample_output = {} 127 for f in feature_combos: --> 128 sample_output.update(f.extract(df)) 129 return sample_output

File /media/gambino/students_workdir/nynke/miniconda3/envs/bento/lib/python3.8/site-packages/bento/tools/_point_features.py:710, in RipleyStats.extract(self, df) 707 stats = estimator.Hfunction(data=points_geo, radii=radii, mode=\"none\") 709 # Max value of the L-function --> 710 l_max = max(stats) 712 # Max and min value of the gradient of L 713 ripley_smooth = pd.Series(stats).rolling(5).mean()

ValueError: max() arg is an empty sequence" }

[ckmah]

No longer relevant after v2.1 refactor.