scverse / pertpy

Perturbation Analysis in the scverse ecosystem.
https://pertpy.readthedocs.io/en/latest/
MIT License
92 stars 19 forks source link

Augur sklearn warning #580

Closed namsaraeva closed 2 months ago

namsaraeva commented 2 months ago

Report

When running adata_15, results_15 = ag_rfc.predict(data_15, random_state=None, n_threads=4) I got this warning (1000s of them actually):

[/Users/altananamsaraeva/miniconda3/envs/pertpy/lib/python3.11/site-packages/sklearn/model_selection/_validation.py:794](https://file+.vscode-resource.vscode-cdn.net/Users/altananamsaraeva/miniconda3/envs/pertpy/lib/python3.11/site-packages/sklearn/model_selection/_validation.py:794): UserWarning: Scoring failed. The score on this train-test partition for these parameters will be set to nan. Details: 
Traceback (most recent call last):
  File "/Users/altananamsaraeva/miniconda3/envs/pertpy/lib/python3.11/site-packages/sklearn/metrics/_scorer.py", line 115, in __call__
    score = scorer._score(cached_call, estimator, *args, **kwargs)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/altananamsaraeva/miniconda3/envs/pertpy/lib/python3.11/site-packages/sklearn/metrics/_scorer.py", line 282, in _score
    return self._sign * self._score_func(y_true, y_pred, **self._kwargs)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: roc_auc_score() got an unexpected keyword argument 'response_method'

Version information

No response

namsaraeva commented 2 months ago

Not sure if this error has something to do with the warning above (which was not present last week):

{
    "name": "StopIteration",
    "message": "",
    "stack": "---------------------------------------------------------------------------
StopIteration                             Traceback (most recent call last)
Cell In[8], line 1
----> 1 ag_rfc.plot_dp_scatter(pvals)

File ~/Packages/pertpy/pertpy/tools/_augur.py:1008, in Augur.plot_dp_scatter(self, results, top_n, ax, return_figure)
    985 \"\"\"Plot scatterplot of differential prioritization.
    986 Args:
    987     results: Results after running differential prioritization.
   (...)
   1005     >>> ag_rfc.plot_dp_scatter(pvals)
   1006 \"\"\"
   1007 x = results[\"mean_augur_score1\"]
-> 1008 y = results[\"mean_augur_score2\"]
   1010 if ax is None:
   1011     fig, ax = plt.subplots()

File ~/miniconda3/envs/pertpy/lib/python3.11/site-packages/matplotlib/__init__.py:1442, in _preprocess_data.<locals>.inner(ax, data, *args, **kwargs)
   1439 @functools.wraps(func)
   1440 def inner(ax, *args, data=None, **kwargs):
   1441     if data is None:
-> 1442         return func(ax, *map(sanitize_sequence, args), **kwargs)
   1444     bound = new_sig.bind(ax, *args, **kwargs)
   1445     auto_label = (bound.arguments.get(label_namer)
   1446                   or bound.kwargs.get(label_namer))

File ~/miniconda3/envs/pertpy/lib/python3.11/site-packages/matplotlib/axes/_axes.py:4602, in Axes.scatter(self, x, y, s, c, marker, cmap, norm, vmin, vmax, alpha, linewidths, edgecolors, plotnonfinite, **kwargs)
   4599 if edgecolors is None:
   4600     orig_edgecolor = kwargs.get('edgecolor', None)
   4601 c, colors, edgecolors = \\
-> 4602     self._parse_scatter_color_args(
   4603         c, edgecolors, kwargs, x.size,
   4604         get_next_color_func=self._get_patches_for_fill.get_next_color)
   4606 if plotnonfinite and colors is None:
   4607     c = np.ma.masked_invalid(c)

File ~/miniconda3/envs/pertpy/lib/python3.11/site-packages/matplotlib/axes/_axes.py:4400, in Axes._parse_scatter_color_args(c, edgecolors, kwargs, xsize, get_next_color_func)
   4393 if c is None:
   4394     c = (facecolors if facecolors is not None
   4395          else \"b\" if mpl.rcParams['_internal.classic_mode']
   4396          else get_next_color_func())
   4397 c_is_string_or_strings = (
   4398     isinstance(c, str)
   4399     or (np.iterable(c) and len(c) > 0
-> 4400         and isinstance(cbook._safe_first_finite(c), str)))
   4402 def invalid_shape_exception(csize, xsize):
   4403     return ValueError(
   4404         f\"'c' argument has {csize} elements, which is inconsistent \"
   4405         f\"with 'x' and 'y' with size {xsize}.\")

File ~/miniconda3/envs/pertpy/lib/python3.11/site-packages/matplotlib/cbook/__init__.py:1715, in _safe_first_finite(obj, skip_nonfinite)
   1712     raise RuntimeError(\"matplotlib does not \"
   1713                        \"support generators as input\")
   1714 else:
-> 1715     return next(val for val in obj if safe_isfinite(val))

StopIteration: "
}
namsaraeva commented 2 months ago

There is no problem with plot_important_features() and it does not produce the sklearn warning.

But

plot_dp_scatter()
plot_lollipop()
plot_scatterplot()

produce empty plots and all throw the sklearn warning. Seems like it breaks the thing.

Zethson commented 2 months ago

https://scikit-learn.org/stable/modules/generated/sklearn.metrics.make_scorer.html exists since version 1.4.0. Your scikit-learn is likely <1.4.0 right? Please upgrade scikit-learn