Closed tabedzki closed 2 weeks ago
https://spikeinterface.readthedocs.io/en/latest/development/development.html
The development docs explain you have to have datalad and git-annex. They are so OS specific we can't automate it at this point,
Quite possibly a PEBKAC situation issue on my end: when reading the "How to build the Documentation", it didn't specify that I needed the other installations after doing pip install -e '.[dev]'
to build the documentation, so I just did a naive installation. Thanks for pointing out what I missed.
I think it is useful for us to know though if people are missing it. We likely need to guide the reader to all the sections in a better way. Thanks for letting us know that the flow isn't necessarily intuitive!
Since datalad has a universal installer, is it safe to use that as part of the installation? What might be some reasons not to include it?
pip install datalad-installer
datalad-installer git-annex -m datalad/packages
pip install datalad
It's not quite that simple. You can look at our CI action that does this for all OSes. (@h-mayorquin got it working so he can comment more!) He originally had all of this in the docs, but we had some versioning issues before the hackathon, so I removed it, but I think he found a new way to install so maybe we put it back into the docs? Not sure.
Hello,
I can't reproduce the error with sphinxcontrib.jquery
. It's a dependency of sphinx_rtd_theme
so, for me, gets installed with that. What OS are you using? And what version of sphinx_rtd_theme
is installed?
I do agree with the PR though - seems like a bad idea to rely on a theme package to install a package required for the docs search.
I am using Python 3.12.4 and the neofetch results are below. I did not install the package separately from the spikeinterface installation process. I created a new venv to install spikeinterface.
(base) ➜ spikeinterface git:(main) ✗ git remote --verbose
origin git@github.com:SpikeInterface/spikeinterface.git (fetch)
origin git@github.com:SpikeInterface/spikeinterface.git (push)
(base) ➜ spikeinterface git:(main) ✗ git pull
Already up to date.
➜ spikeinterface git:(main) ✗ uv pip list
Package Version Editable project location
----------------------------- ------------ ---------------------------------
aiobotocore 2.13.2
aiohappyeyeballs 2.3.5
aiohttp 3.10.3
aioitertools 0.11.0
aiosignal 1.3.1
alabaster 1.0.0
altair 5.4.0
annexremote 1.6.5
asciitree 0.3.3
asttokens 2.4.1
attrs 24.2.0
babel 2.16.0
black 24.8.0
boto3 1.34.131
botocore 1.34.131
cbor2 5.6.4
certifi 2024.7.4
cffi 1.17.0
cfgv 3.4.0
chardet 5.2.0
charset-normalizer 3.3.2
click 8.1.7
cloudpickle 3.0.0
colorcet 3.1.0
colorlog 6.8.2
contourpy 1.2.1
coverage 7.6.1
cryptography 43.0.0
cycler 0.12.1
dask 2024.8.0
datalad 1.1.3
decorator 5.1.1
distlib 0.3.8
distributed 2024.8.0
distro 1.9.0
docutils 0.21.2
elephant 1.1.0
et-xmlfile 1.1.0
executing 2.0.1
fasteners 0.19
figurl 0.2.22
filelock 3.15.4
flake8 7.1.1
fonttools 4.53.1
frozenlist 1.4.1
fsspec 2024.6.1
globus-sdk 3.44.0
graphviz 0.20.3
greenlet 3.0.3
h5py 3.11.0
hdbscan 0.8.38.post1
hdmf 3.14.3
huggingface-hub 0.24.5
humanize 4.10.0
ibl-neuropixel 1.2.0
iblatlas 0.5.3
ibllib 2.38.0
iblutil 1.12.1
identify 2.6.0
idna 3.7
imagecodecs 2024.6.1
imageio 2.35.0
imagesize 1.4.1
iniconfig 2.0.0
ipython 8.26.0
iso8601 2.1.0
jaraco-classes 3.4.0
jaraco-context 5.3.0
jaraco-functools 4.0.2
jedi 0.19.1
jinja2 3.1.4
jmespath 1.0.1
joblib 1.4.2
jsonschema 4.23.0
jsonschema-specifications 2023.12.1
kachery-cloud 0.4.9
keyring 25.3.0
keyrings-alt 5.0.2
kiwisolver 1.4.5
lazy-loader 0.4
lazy-ops 0.2.0
llvmlite 0.43.0
locket 1.0.0
loky 3.4.1
looseversion 1.3.0
markupsafe 2.1.5
matplotlib 3.9.2
matplotlib-inline 0.1.7
mccabe 0.7.0
mearec 1.9.1
meautility 1.5.1
more-itertools 10.4.0
mpmath 1.3.0
msgpack 1.0.8
mtscomp 1.0.2
multidict 6.0.5
mypy-extensions 1.0.0
narwhals 1.4.1
neo 0.14.0.dev0
neovim 0.3.1
networkx 3.3
nodeenv 1.9.1
nptdms 1.9.0
nptyping 2.5.0
numba 0.60.0
numcodecs 0.13.0
numpy 1.26.4
numpydoc 1.8.0
one-api 2.8.1
opencv-python-headless 4.10.0.84
openpyxl 3.1.5
packaging 24.1
pandas 2.2.2
parso 0.8.4
partd 1.4.2
pathspec 0.12.1
patool 2.3.0
patsy 0.5.6
pexpect 4.9.0
phylib 2.6.0
pillow 10.4.0
platformdirs 4.2.2
pluggy 1.5.0
pooch 1.8.2
pre-commit 3.8.0
probeinterface 0.2.24
prompt-toolkit 3.0.47
psutil 6.0.0
psychofit 1.0.0.post0
ptyprocess 0.7.0
pubnub 8.1.0
pure-eval 0.2.3
pyarrow 17.0.0
pycodestyle 2.12.1
pycparser 2.22
pycryptodomex 3.20.0
pyflakes 3.2.0
pygments 2.18.0
pyjwt 2.9.0
pymde 0.1.18
pynndescent 0.5.13
pynrrd 1.0.0
pynvim 0.5.0
pynwb 2.8.1
pyparsing 3.1.2
pyqt5 5.15.11
pyqt5-qt5 5.15.14
pyqt5-sip 12.15.0
pyqtgraph 0.13.7
pytest 8.3.2
pytest-cov 5.0.0
pytest-dependency 0.6.0
python-dateutil 2.9.0.post0
python-gitlab 4.9.0
pytz 2024.1
pyyaml 6.0.2
quantities 0.15.0
referencing 0.35.1
requests 2.32.3
requests-toolbelt 1.0.0
rpds-py 0.20.0
ruamel-yaml 0.18.6
ruamel-yaml-clib 0.2.8
s3fs 2024.6.1
s3transfer 0.10.2
scikit-image 0.24.0
scikit-learn 1.5.1
scipy 1.14.0
seaborn 0.13.2
setuptools 72.2.0
simplejson 3.19.3
six 1.16.0
slidingrp 1.1.1
snowballstemmer 2.2.0
sortedcontainers 2.4.0
sortingview 0.13.5
sparse 0.15.4
sphinx 8.0.2
sphinx-design 0.6.1
sphinx-gallery 0.17.1
sphinx-rtd-theme 0.5.1
sphinxcontrib-applehelp 2.0.0
sphinxcontrib-devhelp 2.0.0
sphinxcontrib-htmlhelp 2.1.0
sphinxcontrib-jquery 4.1
sphinxcontrib-jsmath 1.0.1
sphinxcontrib-qthelp 2.0.0
sphinxcontrib-serializinghtml 2.0.0
spikeinterface 0.101.1 /Users/ct5868/code/spikeinterface
stack-data 0.6.3
statsmodels 0.14.2
sympy 1.13.2
tabulate 0.9.0
tblib 3.0.0
threadpoolctl 3.5.0
tifffile 2024.8.10
toolz 0.12.1
torch 2.4.0
torchvision 0.19.0
tornado 6.4.1
tqdm 4.66.5
traitlets 5.14.3
tridesclous 1.6.8
typing-extensions 4.12.2
tzdata 2024.1
urllib3 2.2.2
virtualenv 20.26.3
wcwidth 0.2.13
wrapt 1.16.0
xarray 2024.7.0
yarl 1.9.4
zarr 2.17.2
zict
(base) ➜ spikeinterface git:(main) ✗ neofetch
'c.
,xNMM. -----------------------
.OMMMMo OS: macOS 14.6.1 23G93 arm64
OMMM0, Host: Mac15,6
.;loddo:' loolloddol;. Kernel: 23.6.0
cKMMMMMMMMMMNWMMMMMMMMMM0: Uptime: 2 days, 16 hours, 24 mins
.KMMMMMMMMMMMMMMMMMMMMMMMWd. Packages: 149 (brew)
XMMMMMMMMMMMMMMMMMMMMMMMX. Shell: zsh 5.9
;MMMMMMMMMMMMMMMMMMMMMMMM: Resolution: 1920x1080, 1512x982
:MMMMMMMMMMMMMMMMMMMMMMMM: DE: Aqua
.MMMMMMMMMMMMMMMMMMMMMMMMX. WM: Rectangle
kMMMMMMMMMMMMMMMMMMMMMMMMWd. Terminal: iTerm2
.XMMMMMMMMMMMMMMMMMMMMMMMMMMk Terminal Font: Monaco 18
.XMMMMMMMMMMMMMMMMMMMMMMMMK. CPU: Apple M3 Pro
kMMMMMMMMMMMMMMMMMMMMMMd GPU: Apple M3 Pro
;KMMMMMMMWXXWMMMMMMMk. Memory: 5096MiB / 36864MiB
.cooc,. .,coo:.
If we need a minimum of 1.2 for sphinx-rtd-theme
, I think we should pin a minimum version into pyproject.toml
to ensure that it gets install along with the jquery
module.
After specifying the package minimum version locally, I was able to build the documentation but I got the following errors:
Running Sphinx v7.4.7
loading translations [en]... done
Using Sphinx-Gallery to convert rst text blocks to markdown for .ipynb files.
[autosummary] generating autosummary for: api.rst, authors.rst, development/development.rst, get_started/import.rst, get_started/index.rst, get_started/install_sorters.rst, get_started/installation.rst, get_started/quickstart.rst, how_to/analyze_neuropixels.rst, how_to/benchmark_with_hybrid_recordings.rst, ..., tutorials/qualitymetrics/plot_4_curation.rst, tutorials/qualitymetrics/sg_execution_times.rst, tutorials/sg_execution_times.rst, tutorials/waveform_extractor_to_sorting_analyzer.rst, tutorials/widgets/plot_1_rec_gallery.rst, tutorials/widgets/plot_2_sort_gallery.rst, tutorials/widgets/plot_3_waveforms_gallery.rst, tutorials/widgets/plot_4_peaks_gallery.rst, tutorials/widgets/sg_execution_times.rst, whatisnew.rst
generating gallery...
generating gallery for tutorials...
generating gallery for tutorials/core... [100%] plot_6_handle_times.py
generating gallery for tutorials/extractors... [100%] plot_2_working_with_unscaled_traces.py
generating gallery for tutorials/qualitymetrics... [100%] plot_4_curation.py
generating gallery for tutorials/comparison... [100%] plot_5_comparison_sorter_weaknesses.py
generating gallery for tutorials/widgets... [100%] plot_4_peaks_gallery.py
computation time summary:
- ../examples/tutorials/core/plot_4_sorting_analyzer.py: 17.28 sec 0.0 MB
- ../examples/tutorials/qualitymetrics/plot_3_quality_mertics.py: 2.17 sec 0.0 MB
- ../examples/tutorials/widgets/plot_3_waveforms_gallery.py: 1.88 sec 0.0 MB
- ../examples/tutorials/qualitymetrics/plot_4_curation.py: 1.25 sec 0.0 MB
- ../examples/tutorials/widgets/plot_4_peaks_gallery.py: 1.15 sec 0.0 MB
- ../examples/tutorials/extractors/plot_1_read_various_formats.py: 1.04 sec 0.0 MB
- ../examples/tutorials/comparison/generate_erroneous_sorting.py: 0.00 sec 0.0 MB
- ../examples/tutorials/comparison/plot_5_comparison_sorter_weaknesses.py: 0.00 sec 0.0 MB
- ../examples/tutorials/core/plot_1_recording_extractor.py: 0.00 sec 0.0 MB
- ../examples/tutorials/core/plot_2_sorting_extractor.py: 0.00 sec 0.0 MB
- ../examples/tutorials/core/plot_3_handle_probe_info.py: 0.00 sec 0.0 MB
- ../examples/tutorials/core/plot_5_append_concatenate_segments.py: 0.00 sec 0.0 MB
- ../examples/tutorials/core/plot_6_handle_times.py: 0.00 sec 0.0 MB
- ../examples/tutorials/extractors/plot_2_working_with_unscaled_traces.py: 0.00 sec 0.0 MB
- ../examples/tutorials/widgets/plot_1_rec_gallery.py: 0.00 sec 0.0 MB
- ../examples/tutorials/widgets/plot_2_sort_gallery.py: 0.00 sec 0.0 MB
loading intersphinx inventory 'neo' from https://neo.readthedocs.io/en/latest/objects.inv...
loading intersphinx inventory 'probeinterface' from https://probeinterface.readthedocs.io/en/stable/objects.inv...
building [mo]: targets for 0 po files that are out of date
writing output...
building [html]: targets for 112 source files that are out of date
updating environment: [new config] 112 added, 0 changed, 0 removed
<unknown>:310: SyntaxWarning: invalid escape sequence '\m'
<unknown>:382: SyntaxWarning: invalid escape sequence '\m'
<unknown>:310: SyntaxWarning: invalid escape sequence '\m'
<unknown>:382: SyntaxWarning: invalid escape sequence '\m'
<unknown>:298: SyntaxWarning: invalid escape sequence '\m'
<unknown>:370: SyntaxWarning: invalid escape sequence '\m'
reading sources... [100%] whatisnew
/Users/ct5868/code/spikeinterface/src/spikeinterface/core/sortinganalyzer.py:docstring of spikeinterface.core.sortinganalyzer.SortingAnalyzer.merge_units:2: WARNING: Inline interpreted text or phrase reference start-string without end-string.
/Users/ct5868/code/spikeinterface/src/spikeinterface/curation/auto_merge.py:docstring of spikeinterface.curation.auto_merge.get_potential_auto_merge:44: WARNING: Bullet list ends without a blank line; unexpected unindent.
/Users/ct5868/code/spikeinterface/src/spikeinterface/sortingcomponents/motion/motion_utils.py:docstring of spikeinterface.sortingcomponents.motion.motion_utils.Motion:11: ERROR: Unexpected indentation.
/Users/ct5868/code/spikeinterface/doc/modules/sorters.rst:534: WARNING: duplicate label modules/sorters:internal sorters, other instance in /Users/ct5868/code/spikeinterface/doc/modules/sorters.rst
looking for now-outdated files... none found
pickling environment... WARNING: cannot cache unpickable configuration value: 'sphinx_gallery_conf' (because it contains a function, class, or module object)
done
checking consistency... done
preparing documents... done
copying assets...
copying downloadable files... [100%] tutorials/widgets/plot_4_peaks_gallery.zip
copying static files... Copying the source path /Users/ct5868/code/spikeinterface/.venv/lib/python3.12/site-packages/sphinx_rtd_theme/static/css/theme.css to /Users/ct5868/code/spikeinterface/doc/_build/_static/css/theme.css will overwrite data, as a file already exists at the destination path and the content does not match.
Copying the source path /Users/ct5868/code/spikeinterface/.venv/lib/python3.12/site-packages/sphinx_rtd_theme/static/css/badge_only.css to /Users/ct5868/code/spikeinterface/doc/_build/_static/css/badge_only.css will overwrite data, as a file already exists at the destination path and the content does not match.
Copying the source path /Users/ct5868/code/spikeinterface/.venv/lib/python3.12/site-packages/sphinx_rtd_theme/static/js/theme.js to /Users/ct5868/code/spikeinterface/doc/_build/_static/js/theme.js will overwrite data, as a file already exists at the destination path and the content does not match.
done
copying extra files... done
copying assets: done
writing output... [100%] whatisnew
/Users/ct5868/code/spikeinterface/doc/sg_execution_times.rst:53: WARNING: Failed to create a cross reference. A title or caption not found: 'sphx_glr_tutorials_comparison_generate_erroneous_sorting.py'
/Users/ct5868/code/spikeinterface/doc/tutorials/index.rst:344: WARNING: toctree contains reference to document 'tutorials/comparison/generate_erroneous_sorting' that doesn't have a title: no link will be generated
/Users/ct5868/code/spikeinterface/doc/tutorials/index.rst:344: WARNING: toctree contains reference to document 'tutorials/comparison/generate_erroneous_sorting' that doesn't have a title: no link will be generated
/Users/ct5868/code/spikeinterface/doc/tutorials/comparison/sg_execution_times.rst:38: WARNING: Failed to create a cross reference. A title or caption not found: 'sphx_glr_tutorials_comparison_generate_erroneous_sorting.py'
/Users/ct5868/code/spikeinterface/doc/tutorials/index.rst:344: WARNING: toctree contains reference to document 'tutorials/comparison/generate_erroneous_sorting' that doesn't have a title: no link will be generated
/Users/ct5868/code/spikeinterface/doc/tutorials/index.rst:344: WARNING: toctree contains reference to document 'tutorials/comparison/generate_erroneous_sorting' that doesn't have a title: no link will be generated
/Users/ct5868/code/spikeinterface/doc/tutorials/index.rst:344: WARNING: toctree contains reference to document 'tutorials/comparison/generate_erroneous_sorting' that doesn't have a title: no link will be generated
/Users/ct5868/code/spikeinterface/doc/tutorials/index.rst:344: WARNING: toctree contains reference to document 'tutorials/comparison/generate_erroneous_sorting' that doesn't have a title: no link will be generated
/Users/ct5868/code/spikeinterface/doc/tutorials/index.rst:344: WARNING: toctree contains reference to document 'tutorials/comparison/generate_erroneous_sorting' that doesn't have a title: no link will be generated
/Users/ct5868/code/spikeinterface/doc/tutorials/index.rst:344: WARNING: toctree contains reference to document 'tutorials/comparison/generate_erroneous_sorting' that doesn't have a title: no link will be generated
/Users/ct5868/code/spikeinterface/doc/tutorials/index.rst:344: WARNING: toctree contains reference to document 'tutorials/comparison/generate_erroneous_sorting' that doesn't have a title: no link will be generated
/Users/ct5868/code/spikeinterface/doc/tutorials/index.rst:344: WARNING: toctree contains reference to document 'tutorials/comparison/generate_erroneous_sorting' that doesn't have a title: no link will be generated
/Users/ct5868/code/spikeinterface/doc/tutorials/index.rst:312: WARNING: Failed to create a cross reference. A title or caption not found: 'sphx_glr_tutorials_comparison_generate_erroneous_sorting.py'
/Users/ct5868/code/spikeinterface/doc/tutorials/index.rst:344: WARNING: toctree contains reference to document 'tutorials/comparison/generate_erroneous_sorting' that doesn't have a title: no link will be generated
/Users/ct5868/code/spikeinterface/doc/tutorials/index.rst:344: WARNING: toctree contains reference to document 'tutorials/comparison/generate_erroneous_sorting' that doesn't have a title: no link will be generated
/Users/ct5868/code/spikeinterface/doc/tutorials/index.rst:344: WARNING: toctree contains reference to document 'tutorials/comparison/generate_erroneous_sorting' that doesn't
have a title: no link will be generated
/Users/ct5868/code/spikeinterface/doc/tutorials/index.rst:344: WARNING: toctree contains reference to document 'tutorials/comparison/generate_erroneous_sorting' that doesn't have a title: no link will be generated
/Users/ct5868/code/spikeinterface/doc/tutorials/index.rst:344: WARNING: toctree contains reference to document 'tutorials/comparison/generate_erroneous_sorting' that doesn't have a title: no link will be generated
/Users/ct5868/code/spikeinterface/doc/tutorials/index.rst:344: WARNING: toctree contains reference to document 'tutorials/comparison/generate_erroneous_sorting' that doesn't have a title: no link will be generated
/Users/ct5868/code/spikeinterface/doc/tutorials/index.rst:344: WARNING: toctree contains reference to document 'tutorials/comparison/generate_erroneous_sorting' that doesn't have a title: no link will be generated
/Users/ct5868/code/spikeinterface/doc/tutorials/index.rst:344: WARNING: toctree contains reference to document 'tutorials/comparison/generate_erroneous_sorting' that doesn't have a title: no link will be generated
generating indices... genindex py-modindex done
writing additional pages... search done
copying images... [ 23%] how_to/benchmark_with_hybrid_recordings_files/benchmark_with_hybrid_recordings_9_0.p
copying images... [ 24%] how_to/benchmark_with_hybrid_recordings_files/benchmark_with_hybrid_recordings_20_0.
copying images... [ 26%] how_to/benchmark_with_hybrid_recordings_files/benchmark_with_hybrid_recordings_28_1.
copying images... [ 27%] how_to/benchmark_with_hybrid_recordings_files/benchmark_with_hybrid_recordings_37_1.
copying images... [ 88%] tutorials/extractors/images/thumb/sphx_glr_plot_2_working_with_unscaled_traces_thumb
copying images... [ 94%] tutorials/comparison/images/thumb/sphx_glr_plot_5_comparison_sorter_weaknesses_thumb
copying images... [100%] tutorials/widgets/images/thumb/sphx_glr_plot_4_peaks_gallery_thumb.png
dumping search index in English (code: en)... done
dumping object inventory... done
Sphinx-Gallery successfully executed 6 out of 6 files subselected by:
gallery_conf["filename_pattern"] = '/plot'
gallery_conf["ignore_pattern"] = '/generate_*'
after excluding 9 files that had previously been run (based on MD5).
embedding documentation hyperlinks...
embedding documentation hyperlinks for tutorials... [100%] plot_1_rec_gallery.html
build succeeded, 26 warnings.
I think specifying the minimum sphinx-rtd-theme is the better move. Their official rec is to actually pin your docs to a specific version. So I think adding in minimums is a step in the right direction and will help new developers that want to build locally not run into an issue like this!
I don't see errors. I see warnings. Unfortunately the warnings typically build up because we are all a bit lax on confirming any code changes don't generate warnings. So Alessio has cleaned up the warnings once. I cleaned up the warnings once (with a big assist from Alessio), and I think Chris did a cleanup once. So it's just something we do every once in a while.
It's one error but not one that prevents compilation.
/Users/ct5868/code/spikeinterface/src/spikeinterface/sortingcomponents/motion/motion_utils.py:docstring of spikeinterface.sortingcomponents.motion.motion_utils.Motion:11: ERROR: Unexpected indentation.
I've gone ahead and added the minimum version for sphinx-rtd-theme
over in #3307
Those are a pain to fix. RTD/Sphinx expects very rigid indentation in docstrings. and since we do some docstring injection it can be hard to trackdown where the issue actually is. This once looks pretty clear, but it's not always. I think Chris's project with the numpydoc will hopefully help us prevent those in the future, but he's in the process of cleaning up all the docs so the linter doesn't explode. I doubt it would become an official linter, but we could always put it in the developer docs to run the linter to try to encourage good behavior moving foward.
Ahh okay. Thanks for explaining a bit of the history and dependency reliance. Happy to have been able to help on this
Steps to reproduce (Python 3.12.4):
gh repo clone spikeinterface/spikeinterface && cd spikeinterface
uv venv && . .venv/bin/activate && uv pip install -e '.[dev]'
sphinx-build -b html doc ./doc/_build/
Error 1:
Steps to remediate:
sphinxcontrib.jquery
to thedocs
list withinpyproject.toml
around line 185uv pip install -e '.[dev]' --no-cache
Re-running
sphinx-build
Error/Warning Stack (edit: some of the original warnings I posted were removed by installing
git annex
. Is there a way to denotegit-annex
needs to be installed for thepip install -e '.[dev]'
?):