search

tslearn-team / tslearn

The machine learning toolkit for time series analysis in Python
https://tslearn.readthedocs.io
BSD 2-Clause "Simplified" License
2.92k stars 342 forks source link

Fix readthedocs failing test #518

Closed YannCabanes closed 4 months ago

YannCabanes commented 5 months ago

The readthedocs continous integration test is failing on the main branch with the following error message:

/home/docs/checkouts/readthedocs.org/user_builds/tslearn/checkouts/518/docs/user_guide/dtw.rst:: WARNING: image file not readable: ../_images/sphx_glr_plot_dtw_thumb.svg
/home/docs/checkouts/readthedocs.org/user_builds/tslearn/checkouts/518/docs/user_guide/early.rst:66: WARNING: image file not readable: ../_images/sphx_glr_plot_early_classification_002.svg
/home/docs/checkouts/readthedocs.org/user_builds/tslearn/checkouts/518/docs/user_guide/matrix-profile.rst:12: WARNING: image file not readable: ../_images/sphx_glr_plot_distance_and_matrix_profile_001.svg
/home/docs/checkouts/readthedocs.org/user_builds/tslearn/checkouts/518/docs/user_guide/shapelets.rst:30: WARNING: image file not readable: ../_images/sphx_glr_plot_shapelet_locations_001.svg
/home/docs/checkouts/readthedocs.org/user_builds/tslearn/checkouts/518/docs/user_guide/shapelets.rst:58: WARNING: image file not readable: ../_images/sphx_glr_plot_shapelet_distances_001.svg
looking for now-outdated files... none found
pickling environment... failed

Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/tslearn/envs/518/lib/python3.8/site-packages/sphinx/cmd/build.py", line 290, in build_main
    app.build(args.force_all, args.filenames)
  File "/home/docs/checkouts/readthedocs.org/user_builds/tslearn/envs/518/lib/python3.8/site-packages/sphinx/application.py", line 351, in build
    self.builder.build_update()
  File "/home/docs/checkouts/readthedocs.org/user_builds/tslearn/envs/518/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 290, in build_update
    self.build(to_build,
  File "/home/docs/checkouts/readthedocs.org/user_builds/tslearn/envs/518/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 327, in build
    pickle.dump(self.env, f, pickle.HIGHEST_PROTOCOL)
_pickle.PicklingError: Can't pickle <class 'matplotlib_svg_scraper'>: attribute lookup matplotlib_svg_scraper on builtins failed

Exception occurred:
  File "/home/docs/checkouts/readthedocs.org/user_builds/tslearn/envs/518/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 327, in build
    pickle.dump(self.env, f, pickle.HIGHEST_PROTOCOL)
_pickle.PicklingError: Can't pickle <class 'matplotlib_svg_scraper'>: attribute lookup matplotlib_svg_scraper on builtins failed
The full traceback has been saved in /tmp/sphinx-err-psikz0d_.log, if you want to report the issue to the developers.
Please also report this if it was a user error, so that a better error message can be provided next time.
A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!
YannCabanes commented 5 months ago

The readthedocs test is failing with the following error message:

/home/docs/checkouts/readthedocs.org/user_builds/tslearn/checkouts/518/docs/auto_examples/metrics/plot_lcss.rst:22: WARNING: Title underline too short.

Longest Common Subsequence
===============
/home/docs/checkouts/readthedocs.org/user_builds/tslearn/checkouts/518/docs/auto_examples/metrics/plot_lcss_custom_metric.rst:22: WARNING: Title underline too short.

Longest Commom Subsequence with a custom distance metric
=============================================
/home/docs/checkouts/readthedocs.org/user_builds/tslearn/checkouts/518/docs/user_guide/dtw.rst:: WARNING: image file not readable: ../_images/sphx_glr_plot_dtw_thumb.svg
/home/docs/checkouts/readthedocs.org/user_builds/tslearn/checkouts/518/docs/user_guide/early.rst:66: WARNING: image file not readable: ../_images/sphx_glr_plot_early_classification_002.svg
/home/docs/checkouts/readthedocs.org/user_builds/tslearn/checkouts/518/docs/user_guide/matrix-profile.rst:12: WARNING: image file not readable: ../_images/sphx_glr_plot_distance_and_matrix_profile_001.svg
/home/docs/checkouts/readthedocs.org/user_builds/tslearn/checkouts/518/docs/user_guide/shapelets.rst:30: WARNING: image file not readable: ../_images/sphx_glr_plot_shapelet_locations_001.svg
/home/docs/checkouts/readthedocs.org/user_builds/tslearn/checkouts/518/docs/user_guide/shapelets.rst:58: WARNING: image file not readable: ../_images/sphx_glr_plot_shapelet_distances_001.svg
looking for now-outdated files... none found
pickling environment... failed

Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/tslearn/envs/518/lib/python3.8/site-packages/sphinx/cmd/build.py", line 290, in build_main
    app.build(args.force_all, args.filenames)
  File "/home/docs/checkouts/readthedocs.org/user_builds/tslearn/envs/518/lib/python3.8/site-packages/sphinx/application.py", line 351, in build
    self.builder.build_update()
  File "/home/docs/checkouts/readthedocs.org/user_builds/tslearn/envs/518/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 290, in build_update
    self.build(to_build,
  File "/home/docs/checkouts/readthedocs.org/user_builds/tslearn/envs/518/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 327, in build
    pickle.dump(self.env, f, pickle.HIGHEST_PROTOCOL)
_pickle.PicklingError: Can't pickle <class 'matplotlib_svg_scraper'>: attribute lookup matplotlib_svg_scraper on builtins failed

Exception occurred:
  File "/home/docs/checkouts/readthedocs.org/user_builds/tslearn/envs/518/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 327, in build
    pickle.dump(self.env, f, pickle.HIGHEST_PROTOCOL)
_pickle.PicklingError: Can't pickle <class 'matplotlib_svg_scraper'>: attribute lookup matplotlib_svg_scraper on builtins failed
YannCabanes commented 5 months ago

There is still the error:

/home/docs/checkouts/readthedocs.org/user_builds/tslearn/checkouts/518/docs/user_guide/dtw.rst:: WARNING: image file not readable: ../_images/sphx_glr_plot_dtw_thumb.svg
/home/docs/checkouts/readthedocs.org/user_builds/tslearn/checkouts/518/docs/user_guide/early.rst:66: WARNING: image file not readable: ../_images/sphx_glr_plot_early_classification_002.svg
/home/docs/checkouts/readthedocs.org/user_builds/tslearn/checkouts/518/docs/user_guide/matrix-profile.rst:12: WARNING: image file not readable: ../_images/sphx_glr_plot_distance_and_matrix_profile_001.svg
/home/docs/checkouts/readthedocs.org/user_builds/tslearn/checkouts/518/docs/user_guide/shapelets.rst:30: WARNING: image file not readable: ../_images/sphx_glr_plot_shapelet_locations_001.svg
/home/docs/checkouts/readthedocs.org/user_builds/tslearn/checkouts/518/docs/user_guide/shapelets.rst:58: WARNING: image file not readable: ../_images/sphx_glr_plot_shapelet_distances_001.svg
looking for now-outdated files... none found
pickling environment... failed

Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/tslearn/envs/518/lib/python3.8/site-packages/sphinx/cmd/build.py", line 290, in build_main
    app.build(args.force_all, args.filenames)
  File "/home/docs/checkouts/readthedocs.org/user_builds/tslearn/envs/518/lib/python3.8/site-packages/sphinx/application.py", line 351, in build
    self.builder.build_update()
  File "/home/docs/checkouts/readthedocs.org/user_builds/tslearn/envs/518/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 290, in build_update
    self.build(to_build,
  File "/home/docs/checkouts/readthedocs.org/user_builds/tslearn/envs/518/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 327, in build
    pickle.dump(self.env, f, pickle.HIGHEST_PROTOCOL)
_pickle.PicklingError: Can't pickle <class 'matplotlib_svg_scraper'>: attribute lookup matplotlib_svg_scraper on builtins failed
YannCabanes commented 5 months ago

This error might be related to the latest matplotlib version 3.9.0 released on May 15, 2024:

YannCabanes commented 5 months ago

Using the Python 3.8 and Matplotlib 3.8.0, we obtain the following error message: ERROR: No matching distribution found for matplotlib==3.8.0

YannCabanes commented 5 months ago

We still have the same error message:

/home/docs/checkouts/readthedocs.org/user_builds/tslearn/checkouts/518/docs/user_guide/dtw.rst:: WARNING: image file not readable: ../_images/sphx_glr_plot_dtw_thumb.svg
/home/docs/checkouts/readthedocs.org/user_builds/tslearn/checkouts/518/docs/user_guide/early.rst:66: WARNING: image file not readable: ../_images/sphx_glr_plot_early_classification_002.svg
/home/docs/checkouts/readthedocs.org/user_builds/tslearn/checkouts/518/docs/user_guide/matrix-profile.rst:12: WARNING: image file not readable: ../_images/sphx_glr_plot_distance_and_matrix_profile_001.svg
/home/docs/checkouts/readthedocs.org/user_builds/tslearn/checkouts/518/docs/user_guide/shapelets.rst:30: WARNING: image file not readable: ../_images/sphx_glr_plot_shapelet_locations_001.svg
/home/docs/checkouts/readthedocs.org/user_builds/tslearn/checkouts/518/docs/user_guide/shapelets.rst:58: WARNING: image file not readable: ../_images/sphx_glr_plot_shapelet_distances_001.svg
looking for now-outdated files... none found
pickling environment... failed

Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/tslearn/envs/518/lib/python3.8/site-packages/sphinx/cmd/build.py", line 290, in build_main
    app.build(args.force_all, args.filenames)
  File "/home/docs/checkouts/readthedocs.org/user_builds/tslearn/envs/518/lib/python3.8/site-packages/sphinx/application.py", line 351, in build
    self.builder.build_update()
  File "/home/docs/checkouts/readthedocs.org/user_builds/tslearn/envs/518/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 290, in build_update
    self.build(to_build,
  File "/home/docs/checkouts/readthedocs.org/user_builds/tslearn/envs/518/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 327, in build
    pickle.dump(self.env, f, pickle.HIGHEST_PROTOCOL)
_pickle.PicklingError: Can't pickle <class 'matplotlib_svg_scraper'>: attribute lookup matplotlib_svg_scraper on builtins failed

Exception occurred:
  File "/home/docs/checkouts/readthedocs.org/user_builds/tslearn/envs/518/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 327, in build
    pickle.dump(self.env, f, pickle.HIGHEST_PROTOCOL)
_pickle.PicklingError: Can't pickle <class 'matplotlib_svg_scraper'>: attribute lookup matplotlib_svg_scraper on builtins failed
The full traceback has been saved in /tmp/sphinx-err-psikz0d_.log, if you want to report the issue to the developers.
Please also report this if it was a user error, so that a better error message can be provided next time.
A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!

Therefore this error message is probably not related to the release of Matplotlib 3.9.0.

YannCabanes commented 5 months ago

The error message might be related to the release of the version 7.3.7 of the package Sphinx on April 19, 2024: https://pypi.org/project/Sphinx/#history https://www.sphinx-doc.org/en/master/internals/release-process.html

YannCabanes commented 5 months ago

The version 0.9.4 of the package nbsphinx has been released on May 6, 2024: https://pypi.org/project/nbsphinx/#history

YannCabanes commented 5 months ago

The version 0.16.0 of the package sphinx-gallery has been released on April 27, 2024: https://pypi.org/project/sphinx-gallery/#history

YannCabanes commented 5 months ago

The version 1.7.0 of the package numpydoc has been released on March 28, 2024: https://pypi.org/project/numpydoc/#history

YannCabanes commented 5 months ago

The version 2.3.1 of torch has been released on June 5, 2024: https://pypi.org/project/torch/#history

YannCabanes commented 5 months ago

The version 2.18.0 of the package Pygments has been released on May 4, 2024: https://pypi.org/project/Pygments/#history

YannCabanes commented 5 months ago

The version 2.16.1 of tensorflow has been released on March 8, 2024: https://pypi.org/project/tensorflow/#history

YannCabanes commented 5 months ago

The version 10.3.0 of the package pillow has been released on April 1, 2024: https://pypi.org/project/pillow/#history

YannCabanes commented 5 months ago

The version 6.29.4 of the package ipykernel has been released on March 27, 2024: https://pypi.org/project/ipykernel/#history

YannCabanes commented 5 months ago

The version 0.60.0 of the package numba has been released on June 13, 2024: https://pypi.org/project/numba/#history

YannCabanes commented 5 months ago

The version 1.5.0 of scikit-learn has been released on May 21, 2024: https://pypi.org/project/scikit-learn/#history

YannCabanes commented 5 months ago

The version 1.14.0 of scipy has been released on June 24, 2024: https://pypi.org/project/scipy/#history

YannCabanes commented 5 months ago

The version 2.0.0 of numpy has been released on June 16, 2024: https://pypi.org/project/numpy/#history

YannCabanes commented 5 months ago

The version 3.0.10 of Cython has been released on March 30, 2024: https://pypi.org/project/Cython/#history

YannCabanes commented 5 months ago

There is still the same error message in the continuous integration test readthedocs.

YannCabanes commented 5 months ago

The following error message is obtained: ERROR: No matching distribution found for sphinx==7.2.6

YannCabanes commented 5 months ago

Using the version 1.6.0 of the package numpydoc, there is still the same error in the continuous integration test readthedocs: _pickle.PicklingError: Can't pickle <class 'matplotlib_svg_scraper'>: attribute lookup matplotlib_svg_scraper on builtins failed

YannCabanes commented 5 months ago

Using the version 0.15.0 of the package sphinx-gallery, the readthedocs continuous integration test succeeded!

The error seemed to be related to the following code from the file conf.py from the folder docs:

from sphinx_gallery.scrapers import matplotlib_scraper

class matplotlib_svg_scraper(object):

    def __repr__(self):
        return self.__class__.__name__

    def __call__(self, *args, **kwargs):
        return matplotlib_scraper(*args, format='svg', **kwargs)
YannCabanes commented 5 months ago

The function sphinx_gallery.scrapers.matplotlib_scraper has been updated in the version 0.16.0 of sphinx_gallery:

YannCabanes commented 5 months ago

Here is the documentation of the module sphinx_gallery.scrapers: https://sphinx-gallery.github.io/stable/gen_modules/sphinx_gallery.scrapers.html

YannCabanes commented 5 months ago

In the Changelog of sphinx-gallery: https://github.com/sphinx-gallery/sphinx-gallery/blob/master/CHANGES.rst it is written for the version 0.16.0 that:

Sphinx 7.3.0 and above changed caching and serialization checks. 
Now instead of passing instantiated classes like ResetArgv(), classes like FileNameSortKey, 
or callables like notebook_modification_function in sphinx_gallery_conf, 
you should pass fully qualified name strings to classes or callables. 
If you change to using name strings, you can simply use a function as the use of classes to ensure 
a stable __repr__ would be redundant.
YannCabanes commented 5 months ago

Useful links to understand how to use matplotlib_scraper from sphinx_gallery.scrapers :

YannCabanes commented 5 months ago

The readthedocs test failed with the following error message:


Running Sphinx v7.1.2
making output directory... done

Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/tslearn/envs/518/lib/python3.8/site-packages/sphinx_gallery/gen_rst.py", line 1573, in _get_callables
    what = importlib.import_module(what)
  File "/home/docs/.asdf/installs/python/3.8.19/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 973, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'matplotlib_svg_scraper'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/tslearn/envs/518/lib/python3.8/site-packages/sphinx/cmd/build.py", line 285, in build_main
    app = Sphinx(args.sourcedir, args.confdir, args.outputdir,
  File "/home/docs/checkouts/readthedocs.org/user_builds/tslearn/envs/518/lib/python3.8/site-packages/sphinx/application.py", line 253, in __init__
    self.events.emit('config-inited', self.config)
  File "/home/docs/checkouts/readthedocs.org/user_builds/tslearn/envs/518/lib/python3.8/site-packages/sphinx/events.py", line 96, in emit
    results.append(listener.handler(self.app, *args))
  File "/home/docs/checkouts/readthedocs.org/user_builds/tslearn/envs/518/lib/python3.8/site-packages/sphinx_gallery/gen_gallery.py", line 1491, in fill_gallery_conf_defaults
    new_sphinx_gallery_conf = _fill_gallery_conf_defaults(
  File "/home/docs/checkouts/readthedocs.org/user_builds/tslearn/envs/518/lib/python3.8/site-packages/sphinx_gallery/gen_gallery.py", line 239, in _fill_gallery_conf_defaults
    _get_callables(gallery_conf, key)
  File "/home/docs/checkouts/readthedocs.org/user_builds/tslearn/envs/518/lib/python3.8/site-packages/sphinx_gallery/gen_rst.py", line 1577, in _get_callables
    raise ConfigError(f"Unknown string option for {readable}")
sphinx.errors.ConfigError: Unknown string option for image_scrapers[0]='matplotlib_svg_scraper'

Configuration error:
Unknown string option for image_scrapers[0]='matplotlib_svg_scraper'
YannCabanes commented 5 months ago

The same error message is obtained:

    raise ConfigError(
sphinx.errors.ConfigError: Unknown string option image_scrapers[0]='tslearn.docs.utils.matplotlib_svg_scraper' when importing matplotlib_svg_scraper from tslearn.docs.utils

Configuration error:
Unknown string option image_scrapers[0]='tslearn.docs.utils.matplotlib_svg_scraper' when importing matplotlib_svg_scraper from tslearn.docs.utils
YannCabanes commented 5 months ago

This error is raised by the file: https://github.com/sphinx-gallery/sphinx-gallery/blob/master/sphinx_gallery/gen_rst.py

YannCabanes commented 5 months ago

The readthedocs test failed with the following error message:

looking for now-outdated files... none found
pickling environment... failed

Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/tslearn/envs/518/lib/python3.8/site-packages/sphinx/cmd/build.py", line 290, in build_main
    app.build(args.force_all, args.filenames)
  File "/home/docs/checkouts/readthedocs.org/user_builds/tslearn/envs/518/lib/python3.8/site-packages/sphinx/application.py", line 351, in build
    self.builder.build_update()
  File "/home/docs/checkouts/readthedocs.org/user_builds/tslearn/envs/518/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 290, in build_update
    self.build(to_build,
  File "/home/docs/checkouts/readthedocs.org/user_builds/tslearn/envs/518/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 327, in build
    pickle.dump(self.env, f, pickle.HIGHEST_PROTOCOL)
_pickle.PicklingError: Can't pickle <class 'MatplotlibSVGScraper'>: attribute lookup MatplotlibSVGScraper on builtins failed

Exception occurred:
  File "/home/docs/checkouts/readthedocs.org/user_builds/tslearn/envs/518/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 327, in build
    pickle.dump(self.env, f, pickle.HIGHEST_PROTOCOL)
_pickle.PicklingError: Can't pickle <class 'MatplotlibSVGScraper'>: attribute lookup MatplotlibSVGScraper on builtins failed
The full traceback has been saved in /tmp/sphinx-err-svd6vl4z.log, if you want to report the issue to the developers.
Please also report this if it was a user error, so that a better error message can be provided next time.
A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!
YannCabanes commented 5 months ago

Here is a link for sphinx-gallery image scrapers: https://sphinx-gallery.github.io/stable/configuration.html#image-scrapers

YannCabanes commented 5 months ago

Another useful link: https://www.sphinx-doc.org/en/master/usage/configuration.html

The term “fully-qualified name” refers to a string that names an importable Python object inside a module; 
for example, the FQN "sphinx.builders.Builder" means the Builder class in the sphinx.builders module.
YannCabanes commented 5 months ago

All test have passed (except codecov which was cancelled after 60 minutes).

rtavenar commented 5 months ago

Indeed, the docs now build, but generated images are no longer svg, which is a pity imho.

YannCabanes commented 4 months ago

Thank you @rtavenar, I decided to temporarily downgrade sphinx-gallery to the version 0.15.0 and to try to fix the compatibility of tslearn's codes with sphinx-gallery 0.16.0 in a future pull request.