`Literal` inconsistent processing

[mhostetter]

In my library I discovered instances where Literal[] would be represented in a strange way. I extracted two functions to demonstrate in one of my foo packages. foo-275.zip

Good

Bad

Expected output: "binary" | "bipolar" = "bipolar"

Output

Python 3.8.10 Sphinx 6.2.1 Sphinx Immaterial 0.11.6

$ sphinx-build -b dirhtml -v docs/ docs/build/
Running Sphinx v6.2.1
loading pickled environment... done
locale_dir /mnt/c/Users/matth/repos/foo-new/docs/locales/en/LC_MESSAGES does not exists
building [mo]: targets for 0 po files that are out of date
writing output... 
building [dirhtml]: targets for 4 source files that are out of date
updating environment: locale_dir /mnt/c/Users/matth/repos/foo-new/docs/locales/en/LC_MESSAGES does not exists
[config changed ('sphinx_immaterial_custom_admonitions')] 4 added, 0 changed, 0 removed
reading sources... [ 25%] api
reading sources... [ 50%] api/foo.barker
reading sources... [ 75%] api/foo.periodogram
reading sources... [100%] index

/mnt/c/Users/matth/repos/foo-new/docs/api.rst.rst:7: WARNING: No top-level Python API group named: 'classes', valid groups are: ['public-members']
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
writing output... [ 25%] api
writing output... [ 50%] api/foo.barker
writing output... [ 75%] api/foo.periodogram
writing output... [100%] index

generating indices... genindex done
writing additional pages... done
copying static files... done
copying extra files... done
dumping search index in English (code: en)... done
dumping object inventory... done
Generating sitemap for 5 pages in /mnt/c/Users/matth/repos/foo-new/docs/build/sitemap.xml
build succeeded, 1 warning.

The HTML pages are in docs/build.

[2bndy5]

I haven't come across this problem, and I have set the default value of a string literal (for a class instance attr instead of a function param).

I have noticed that if the type hints are inherited, then they are displayed exactly as they are in src (using autodoc, not apigen).

[jbms]

The bug is actually in sphinx, namely in sphinx.util.typing.stringify_annotation. It has special handling for typing.Literal but in Python 3.8 Literal isn't available so it is typing_extensions.Literal, which Sphinx doesn't stringify correctly.

The Literal handling modifications to the plain Python domain directives implemented in this theme does handle it, because there is a default mapping from typing_extensions to typing. But by the signature reaches the Python domain directives, the quotes have already been lost.

I didn't investigate exactly why the issue doesn't occur with the other signature, but it appears to be due to the use of | syntax --- that syntax is not supported by Python 3.8, which may somehow lead to the annotations just passing through autodoc without munging, such that they are formatted correctly.

[mhostetter]

Thanks for the help, as always!

I thought maybe updating to Python 3.9 would alleviate this issue. It did not. Then I realized/remembered that I haven't experienced this issue in my other project. After some digging I learned that if I revert back to Sphinx Immaterial 0.11.2, Typing Extensions 4.5.0, and Pydantic v1, it renders correctly.

As soon as I bumped to Sphinx Immaterial 0.11.6, that required Pydantic v2, which required Typing Extensions 4.7.1. I'm guessing the offending code is in Typing Extensions. Maybe the changes in v4.6.0.

Any thoughts on workarounds? It seems I can't use newer versions of this theme since it pins pydantic>=2.0.

[jbms]

I didn't test Python 3.9 but it does not occur with 3.11.

[mhostetter]

Thanks! I can confirm that works for me too.

[2bndy5]

https://github.com/sphinx-doc/sphinx/issues/11473 seems related and might be amended in Sphinx v7.2.0

[2bndy5]

Just tested the foo-275 example with python v3.10, sphinx v7.1.2, and sphinx-immaterial v0.11.7.

Should this remain open? My understanding is that this issue is more subject to dependencies.

[jbms]

This is a sphinx bug and not specific to this theme, and also only affects Python 3.8 which is no longer supported by the latest sphinx, so yes I think it can be closed.

[mhostetter]

Thanks for the help, guys!