JOSS review: basic_example non-functional

[robcaulk]

Hello,

I am one of the reviewers on the JOSS review for this repository https://github.com/openjournals/joss-reviews/issues/6841.

Following the pip installation instructions located in the docs https://pycvi.readthedocs.io/en/latest/index.html, the examples are not runnable. I copy pasted the example (https://pycvi.readthedocs.io/en/latest/examples/basic_usage.html) into a script, but could see that it would not work. It doesn't work because you have a raw import on the import ..utils plot_true_selected function, which is not part of your pypi package since it is not located within the pyci folder.

In any case, ok, forget the pypi install method, I tried to clone the repo and test, and still ran into some issues:

python3 basic_usage.py 
Traceback (most recent call last):
  File "/home/rcaulk/projects/JOSS_reviews/pycvi/examples/basic_usage/basic_usage.py", line 13, in <module>
    from ..utils import plot_true_selected
ImportError: attempted relative import with no known parent package

The current (wrong) solution would be to copy paste utils.py into the basic_example directory. But generally speaking, if this utils.py is integral for users to learn your code, you should include it in your package so that it can be imported as necessary.

Then we run into the next issue, which is that matplotlib is not installed.

Is matplotlib an "extra" that is not a dependency for this library? If it is not required, but integral for users to learn your code, it would be preferable to at least add this as a doc string to the example scripts. But more likely you would add this to your pypi package pip install pycvi-lib[plot]

After pip install matplotlib, I run into:

python3 basic_usage.py 
Traceback (most recent call last):
  File "/home/rcaulk/projects/JOSS_reviews/pycvi/examples/basic_usage/basic_usage.py", line 13, in <module>
    from utils import plot_true_selected
  File "/home/rcaulk/projects/JOSS_reviews/pycvi/examples/basic_usage/utils.py", line 3, in <module>
    from matplotlib.cm import get_cmap
ImportError: cannot import name 'get_cmap' from 'matplotlib.cm' (/home/rcaulk/.venv/lib/python3.11/site-packages/matplotlib/cm.py)

My matplotlib version seems to fit into your pyproject.toml requirements

pip show matplotlib
Name: matplotlib
Version: 3.9.0
Summary: Python plotting package
Home-page: https://matplotlib.org

And the python version version that I am using is 3.11, which also fits into your pyproject.toml specification.

[nglm]

First all of thank you very much @robcaulk for your great observations! Sometimes when you're too deep into your projects, you don't see the really obvious things anymore!

I hope the updates detailed below have fixed all the mentioned issues, but please let me know if there are still some problems!

Some explanations/context of the issues

• There was an optional dependency group "examples" that could be locally installed with for example the command poetry install --with examples. But thanks to your comments, I realised that I should have used extras dependencies that are installable from everywhere instead of group dependencies that are only installable locally. In any case, I really want to have matplotlib outside the necessary requirements of PyCVI.
• The examples scripts were initially designed for a "github audience", using a module-based type of design (e.g. the script basic_usage.py could be run from the package root by running python -m examples.basic_usage, assuming that the user had downloaded the whole package repository). This was more convenient for me at the beginning, but now that you point at it, it is indeed a terrible design choice when used in a documentation. The idea was also that I really didn't want to pollute the example scripts with plot functions (hence the relative import of the utils.py file).

Summary of what has been done to fix it

• Create an "example" extra set of dependencies that contain the necessary dependencies to run the example scripts. This extra set of dependency can be installed by running:

  # using poetry
  poetry add pycvi-lib[examples]
  # using pip or anaconda
  pip install pycvi-lib[examples]

• Switch from a module-based type of example scripts to scripts that follow what a user would actually do, that is:

  • Install PyCVI in their environment, directly with the optional dependencies examples or after manually installing the package related to the specific example (e.g. matplotlib and/or sklearn-extra)
  • Create a script by copying directly or getting inspiration from a given example in the documentation and the utils file.
  • Run their own script in their environment.

• Specify on the README page as well as on the welcome page of the documentation that there are extra steps to follow to be able to run the example script on your computer.

• Specify very clearly in each page of the documentation showcasing an example to carefully follow the instructions in order to run the scripts on their own computers. That is, it is necessary to:

  • First install in their environment PyCVI, but also:
  • The extra dependency group examples for all examples
  • Or manually install specific packages needed for a given example on a basic PyCVI installation.
  • Copy the utils file as well (with a clear link in the documentation to this utils file.)

• Fix the dependency issues between the examples extra dependency group and the rest of the packages in the case of python 3.11 (and check more carefully with other versions as well, use different version constraints for different python versions.) Matplotlib 3.9 was the main culprit, but other dependencies have been checked as well.

• Released a new version of PyCVI and of the documentation (0.1.2).