Docstrings in help pane shown without formatting, first ctrl+i causes error "No module named 'sphinx.ext.jsmath'"

[Mister-Teatime]

Issue Report Checklist

• [x] Searched the issues page for similar reports I'm aware of issues #15609 and #15622. This one looks very similar, except those issues are about older Spyder versions and were reported fixed years ago.
• [x] Read the relevant sections of the Spyder Troubleshooting Guide and followed its advice
• [x] Reproduced the issue after updating with conda update spyder (or pip, if not using Anaconda) No, but via pipenv install, and via pipx, on multiple machines. The issue persists since at least December 2023.
• [-] Could not reproduce inside jupyter qtconsole (if console-related)
• [x] Tried basic troubleshooting (if a bug/error)
  • [x] Restarted Spyder
  • [x] Reset preferences with spyder --reset -- no, but mutliple fresh installations on different machines, including one which did not have Spyder installed previously
  • [-] (not using Anaconda, I use pipenv and pipx) Reinstalled the latest version of Anaconda
  • [-] (Found no other appliccable steps) Tried the other applicable steps from the Troubleshooting Guide
• [x] Completed the Problem Description, Steps to Reproduce and Version sections below

Problem Description

There are no formatted docstrings shown in the help pane (although completion tooltips do work). Spyder reports an that sphinx.ext.jsmath was missing.

What steps reproduce the problem?

1. Launch Spyder 5.5.5 (probably the same for 5.5.x)
2. in the console, import numpy (import numpy as np) and then type np.linspace
3. alternatively, open any Python file, place the cursor on some item with an associated docstring and hit ctrl+I

What is the expected output? What do you see instead?

In the help pane, the docstring should be shown, in a well-formatted way, plus a tooltip with some condensed information.

Instead, an error message pops up:

The following error occurred when calling Sphinx 7.3.7. Incompatible Sphinx version or doc string decoding failed.

Error message: Could not import extension sphinx.ext.jsmath (exception: No module named 'sphinx.ext.jsmath') ...and the help pane shows the following text, in a fixed-width font:

Here you can get help of any object by pressing Ctrl+I in front of it, either on the Editor or the Console.

Help can also be shown automatically after writing a left parenthesis next to an object. You can activate this behavior in Preferences > Help.

Please consider installing Sphinx to get documentation rendered in rich text.

If I then type the opening parenthesis for the arguments, the help pane instead shows me the source code of np.linspace, also in fixed-width font but with syntax highlighting. This includes the call signature and raw docstring -- but that is of course not what is meant to appear there.

### Paste Traceback/Error Below (if applicable)
(error message: see above)
There is no output to the internal console, or to the bash shell from which I launched Spyder, and no traceback.

## Versions

* Spyder version: 5.5.5
* Python version: 3.10.12 64-bit
* Qt version: Qt 5.15.2
* PyQt version: PyQt5 5.15.10
* Operating System name/version: Linux 6.5.0-41-generic (x86_64) 
    * Kubuntu 20.04, to be more precise

### Dependencies

Mandatory:

atomicwrites >=1.2.0 : 1.4.1 (OK) chardet >=2.0.0 : 5.2.0 (OK) cloudpickle >=0.5.0 : 3.0.0 (OK) cookiecutter >=1.6.0 : 2.6.0 (OK) diff_match_patch >=20181111 : 20230430 (OK) intervaltree >=3.0.2 : 3.1.0 (OK) IPython >=8.13.0,<9.0.0,!=8.17.1 : 8.26.0 (OK) jedi >=0.17.2,<0.20.0 : 0.19.1 (OK) jellyfish >=0.7 : 1.0.4 (OK) jsonschema >=3.2.0 : 4.23.0 (OK) keyring >=17.0.0 : 25.2.1 (OK) nbconvert >=4.0 : 7.16.4 (OK) numpydoc >=0.6.0 : 1.7.0 (OK) parso >=0.7.0,<0.9.0 : 0.8.4 (OK) pexpect >=4.4.0 : 4.9.0 (OK) pickleshare >=0.4 : 0.7.5 (OK) psutil >=5.3 : 6.0.0 (OK) pygments >=2.0 : 2.18.0 (OK) pylint >=3.1,<4 : 3.2.5 (OK) pylint_venv >=3.0.2 : 3.0.3 (OK) pyls_spyder >=0.4.0 : 0.4.0 (OK) pylsp >=1.11.0,<1.12.0 : 1.11.0 (OK) pylsp_black >=2.0.0,<3.0.0 : 2.0.0 (OK) qdarkstyle >=3.2.0,<3.3.0 : 3.2.3 (OK) qstylizer >=0.2.2 : 0.2.3 (OK) qtawesome >=1.3.1,<1.4.0 : 1.3.1 (OK) qtconsole >=5.5.1,<5.6.0 : 5.5.2 (OK) qtpy >=2.1.0 : 2.4.1 (OK) rtree >=0.9.7 : 1.3.0 (OK) setuptools >=49.6.0 : 70.3.0 (OK) sphinx >=0.6.6 : 7.3.7 (OK) spyder_kernels >=2.5.2,<2.6.0 : 2.5.2 (OK) textdistance >=4.2.0 : 4.6.2 (OK) three_merge >=0.1.1 : 0.1.1 (OK) watchdog >=0.10.3 : 4.0.1 (OK) xdg >=0.26 : 0.28 (OK) zmq >=24.0.0 : 26.0.3 (OK)

Optional:

cython >=0.21 : None (NOK) matplotlib >=3.0.0 : None (NOK) numpy >=1.7 : None (NOK) pandas >=1.1.1 : None (NOK) scipy >=0.17.0 : None (NOK) sympy >=0.7.3 : None (NOK)

(this is a pipx environment, and contains nothing but Spyder and dependencies. The code I write uses different venvs, per project, which might explain why `import numpy` works even though `numpy` is not installed in the virtual environment)

### Things I've tried before
I've been seeing this issue since January now, and did at some point try to force installation of older sphinx versions. Unfortunately, I did not keep detailed notes, and it took me quite some time without improving the situation, so I will not try to repeat the exercise unless somebody suggests a specific sphinx version to me. I do remember that the error message was different for some versions, but I found no version which worked.

I also tried explicitly installing sphinx into the project environment (i.e. the one which runs the console), but this did not have any effect. The project environment does contain `spyder_kernels`, of course.

[ccordoba12]

Hey @Mister-Teatime, thanks for reporting. Please install sphinxcontrib-jsmath, that should solve your problem.

[ccordoba12]

Closing due to lack of response.

[Mister-Teatime]

Apologies for not getting back in time -- life intervened, and then I remembered to come here 8 minutes after you closed it...

I manually installed sphinxcontrib-jsmath in the pipx environment. In the meantime, it had automatically upgraded to sphinx 8.0.2, I verified that sphinxcontrib-jsmath was installed, and the error was exactly the same.

I just checked one of the pipenvs which have Spyder installed (this one uses sphinx 7.2.6), verified that jsmath is already installed.... (excerpt of output from pipenv graph)

│   ├── sphinx [required: >=5, installed: 7.2.6]
│   │   ├── alabaster [required: >=0.7,<0.8, installed: 0.7.16]
│   │   ├── Babel [required: >=2.9, installed: 2.14.0]
│   │   ├── docutils [required: >=0.18.1,<0.21, installed: 0.20.1]
[...]
│   │   ├── sphinxcontrib-devhelp [required: Any, installed: 1.0.6]
│   │   ├── sphinxcontrib-htmlhelp [required: >=2.0.0, installed: 2.0.5]
│   │   ├── sphinxcontrib-jsmath [required: Any, installed: 1.0.1]
[...]

...and the issue remains the same. So sphinxcontrib-jsmath has correctly been installed in this pipenv, automatically.

I had a hunch that maybe it was somehow colliding with some non-virtualenv spyder installation, so I made sure to pip uninstall spyder, but this did not change anything, either.

Not sure where else to look, at this point. One faint idea: There is a system-wide installation of sphinx 4.3.2, installed via the package manager. This is a remainder from some time ago when I tried to install spyder via the package manager, and ran into the same issues and also manually installed python3-sphinxcontrib-jsmath (v. 1.0.1-3) along with it. Weirdly enough, the repositories for Ubuntu 22.04 don't seem to have a newer version of sphinx available -- but in order for this to matter to my virtual environments, there would need to be something wrong with those, too. I've encountered the same issue on two different computers, via the package manager, via pipenv and via pipx: the jsmath extension is installed, yet I'm receiving this message. I've no idea where I could look for further clues -- any useful hints appreciated.

[Mister-Teatime]

Quick follow-up:

I created a fresh pipenv:

pipenv install spyder-kernels==2.5
Creating a virtualenv for this project...
Pipfile: /<something>/testpipenv/Pipfile
Using default python from /usr/bin/python3 (3.10.12) to create virtualenv...
⠼ Creating virtual environment...created virtual environment CPython3.10.12.final.0-64 in 312ms
[...]
Virtualenv location: /home/<uname>/.local/share/virtualenvs/testpipenv-WurVFNRr
Creating a Pipfile for this project...
[...]
✔ Installation Succeeded
Pipfile.lock not found, creating...
Locking [packages] dependencies...
Building requirements...
Resolving dependencies...
✔ Success!
[...]

I used the new virtualenv as the python interpreter for the spyder that lives in my pipx environment, and got the same issue as before.

Then, I installed some extra things in the new venv:

$ pipenv install numpy
$ pipenv install sphinx

Tried again, to ensure that the sphinx version in the venv is also not to blame. (also made sure that the venv has the same sphinx version as the pipx environment which contains Spyder). Again, no changes to the observed behaviour.

I then installed Spyder itself in the new venv, started that version, entirely within the same, new venv, using the same interpreter for the development as the one that is running spyder itself, and the result is still unchanged.

I also verified that sphinxcontrib-jsmath was in fact installed.

However, then I found the following:

In [4]: import sphinx.ext.jsmath
Traceback (most recent call last):

  Cell In[4], line 1
    import sphinx.ext.jsmath

ModuleNotFoundError: No module named 'sphinx.ext.jsmath'

In [5]: import sphinx.ext.mathjax
In [6]: sphinx.__version__
Out[6]: '8.0.2'

In [7]: from sphinxcontrib import jsmath
In [8]: 

So, sphinxcontrib-jsmath is installed, and everyone confirms this. sphinx.ext.jsmath cannot be imported. sphinx.ext.mathjax has no such issues, though, and autocompletion finds a host of other things I could import from sphinx.ext if I wanted -- just jsmath is not one of them. And sphinxcontrib.jsmath does exist and can be imported.

Just for fun, I tried the same thing with the system python environment, which has sphinx 4.3.2 installed, and exactly the same happens, even though this appears to be a much older sphinx version.

edit:

...aand last night, I got my hands on another laptop with manjaro, and Spyder installed via package manager. All the same libraries are installed, import sphinx.ext.jsmath still fails but Spyder and Sphinx seem to get along just fine -- all docstrings shown as they should, no error messages.

What can I make of all this?

On my end, this looks as if some part of either Spyder or Sphinx did not know where to import jsmath from -- except of course I know that this can't be true because then everybody would have the same issue as I do (edit: including the laptop I tried it on yesterday -- which clearly demonstrates that I have no idea how it's supposed to work in the first place) The issue is however reproducible on my end, on two computers, on Kubuntu and Manjaro, across a pretty large range of sphinx versions.

So maybe the information above helps you have some idea about where the issue might be? I can't be the only person running Spyder on Linux, and it used to work for me last year. So maybe there's something I'm doing differently on my computers than most people? I could try some other python environment manager but since even the system version on Kubuntu shows the same issue, I don't think that could explain it, either.

I guess I could try and evade the issue by creating a sneaky symbolic link to force sphinx.ext.jsmath into existence, but that looks like a very hacky approach, with the potential for intractable side effects given that directly importing it fails even on systems that work as expected, I suspect it might not actually help.

==> all useful hints appreciated. There is a large amount of things I could try, or places I could look, and I'd really like to narrow down the scope a bit.

[ccordoba12]

==> all useful hints appreciated. There is a large amount of things I could try, or places I could look, and I'd really like to narrow down the scope a bit.

Sorry to say it, but we don't have time to help users with complex setups such as yours. That's why recommend people to either use Anaconda or install Spyder in a virtualenv.

In Spyder 6 we'll even provide our own Linux installer. That'd be the simplest way to install Spyder and avoid these headaches. Afterwards, you only need to connect that installation to the env you want to work on by following these instructions:

http://docs.spyder-ide.org/current/faq.html#using-existing-environment

[Mister-Teatime]

Closing due to lack of response.

Would you mind taking another look? Added some more detail.

[ccordoba12]

I know you added more details, that's why I posted an additional comment in https://github.com/spyder-ide/spyder/issues/22282#issuecomment-2299226197.

So, we can't offer you additional support because you're using a complex Python setup that we don't endorse. We're a very small team and don't have time to help users with those setups, sorry.

[Mister-Teatime]

You have a point: I am apparently too confused to notice that you replied (or I replied on a non-refreshed version of the page? I really can't tell, I'm afraid.)

==> all useful hints appreciated. There is a large amount of things I could try, or places I could look, and I'd really like to narrow down the scope a bit.

Sorry to say it, but we don't have time to help users with complex setups such as yours.

I don't get why you think my setup was complex. I reproduced the issue in a single virtualenv with nothing but Spyder plus dependencies in it. Then, I reproduced it on a different machine without any virtualenvs involved, just by installing things from system packages.

That's why recommend people to either use Anaconda or install Spyder in a virtualenv. My Spyder installations are in virtualenvs, for exactly this reason, and I am seeing the issue independent of whether I'm using the python console from the same virtualenv or not.

In Spyder 6 we'll even provide our own Linux installer. That'd be the simplest way to install Spyder and avoid these headaches. Afterwards, you only need to connect that installation to the env you want to work on by following these instructions:

http://docs.spyder-ide.org/current/faq.html#using-existing-environment

That is exactly my usual setup (and thanks for that FAQ because it was very useful to get it to work!)

• Spyder installed in a virtualenv (managed by pipenv but I trust pipenv more about managing virtualenvs than I trust myself in the matter)
• project in a separate virtualenv that has spyder-kernels installed (also via pipenv, of course)

... and because I suspected that there may be some issues with this, I also tried a lot of different variants -- all show the same issue, across multiple version of spyder and sphinx. As a result, despite having no hunch what the issue is, I can pretty much prove that it's not the complexity of my setup.

Since the issue does not appear or disappear with different versions of spyder or sphinx, there is probably something else on two of the computers I use. That is incredibly vague, and if you have no direct idea what could be going on, I was at least hoping for some useful pointers.

Some random ideas:

• Are there any settings/configurations that have the ability to break sphinx functionality, which may have survived from earlier versions, or earlier failed set-ups? I copied some configuration files in ~/.config and ~/.local over from a previous installation (which worked fine, but the machine broke) -- would they have a chance of interfering?
• Is there any way for a system library with an incompatible version to interfere with things in a virtualenv?
• I'd hate to reset all my settings in Spyder because I've tweaked them quite a bit, but is there a chance that some of those could interfere? If yes, which ones?
• Is there a way I could test whether the respective sphinx functionality works correctly without spyder? That would at least clarify whether it's sphinx or spyder that does not work as it should.

[Mister-Teatime]

Just upgraded to Spyder 6.0.0 (via pipx), and after also updating the spyder-kernels package in the project environment, sphinx is working as expected again, and I see docstrings rendered as expected.

I'd still like to know what on earth was going wrong before, but (at least on one of the two machines...) the issue is resolved now.

[ccordoba12]

I think this problem was solved by https://github.com/spyder-ide/spyder/pull/21518, only present in Spyder 6.