parallel mode of sphinx-build could not detect duplicated labels

tk0miya commented 6 years ago

Problem

Sphinx could not detect duplicated labels with -j N option

Procedure to reproduce the problem

Here is my Dockerfile.

FROM tk0miya/sphinx-html

RUN sphinx-quickstart -q -p test -a test test
WORKDIR /docs/test
RUN printf ".. _test:\n\nhello\n=====\n" > index.rst && \
    printf ".. _test:\n\nhello\n=====\n" > foo.rst && \
    printf ".. _test:\n\nhello\n=====\n" > bar.rst && \
    printf ".. _test:\n\nhello\n=====\n" > baz.rst && \
    printf ".. _test:\n\nhello\n=====\n" > qux.rst && \
    printf ".. _test:\n\nhello\n=====\n" > quux.rst
RUN make html
RUN make clean
RUN sphinx-build -b html -j10 . _build/html

Error logs / results

On non-parallel mode, there are 5 warnings for duplicated label.

Step 5/7 : RUN make html
 ---> Running in c0bc5c3452f9
Running Sphinx v1.6.6
making output directory...
loading pickled environment... not yet created
building [mo]: targets for 0 po files that are out of date
building [html]: targets for 6 source files that are out of date
updating environment: 6 added, 0 changed, 0 removed
reading sources... [ 16%] bar
reading sources... [ 33%] baz
reading sources... [ 50%] foo
reading sources... [ 66%] index
reading sources... [ 83%] quux
reading sources... [100%] qux

looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
writing output... [ 16%] bar
/docs/test/baz.rst:4: WARNING: duplicate label test, other instance in /docs/test/bar.rst
/docs/test/foo.rst:4: WARNING: duplicate label test, other instance in /docs/test/baz.rst
/docs/test/index.rst:4: WARNING: duplicate label test, other instance in /docs/test/foo.rst
/docs/test/quux.rst:4: WARNING: duplicate label test, other instance in /docs/test/index.rst
/docs/test/qux.rst:4: WARNING: duplicate label test, other instance in /docs/test/quux.rst
/docs/test/bar.rst: WARNING: document isn't included in any toctree
/docs/test/baz.rst: WARNING: document isn't included in any toctree
/docs/test/foo.rst: WARNING: document isn't included in any toctree
/docs/test/quux.rst: WARNING: document isn't included in any toctree
/docs/test/qux.rst: WARNING: document isn't included in any toctree
writing output... [ 33%] baz
writing output... [ 50%] foo
writing output... [ 66%] index
writing output... [ 83%] quux
writing output... [100%] qux

generating indices... genindex
writing additional pages... search
copying static files... done
copying extra files... done
dumping search index in English (code: en) ... done
dumping object inventory... done
build succeeded, 10 warnings.

Build finished. The HTML pages are in _build/html.

On the other hand, with -j option, there are no warnings.

Step 7/7 : RUN sphinx-build -b html -j10 . _build/html
 ---> Running in 8efe0829c5a4
Running Sphinx v1.6.6
making output directory...
loading pickled environment... not yet created
building [mo]: targets for 0 po files that are out of date
building [html]: targets for 6 source files that are out of date
updating environment: 6 added, 0 changed, 0 removed
reading sources... [ 16%] bar
reading sources... [ 33%] baz
reading sources... [ 50%] foo
reading sources... [ 66%] index
reading sources... [ 83%] quux
reading sources... [100%] qux

waiting for workers...
looking for now-outdated files... none found
pickling environment... done
checking consistency... /docs/test/bar.rst: WARNING: document isn't included in any toctree
/docs/test/baz.rst: WARNING: document isn't included in any toctree
/docs/test/foo.rst: WARNING: document isn't included in any toctree
/docs/test/quux.rst: WARNING: document isn't included in any toctree
/docs/test/qux.rst: WARNING: document isn't included in any toctree
done
preparing documents... done
writing output... [ 20%] baz
writing output... [ 40%] foo
writing output... [ 60%] index
writing output... [ 80%] quux
writing output... [100%] qux

waiting for workers...
generating indices... genindex
writing additional pages... search
copying static files... done
copying extra files... done
dumping search index in English (code: en) ... done
dumping object inventory... done
build succeeded, 5 warnings.

Expected results

Sphinx have to work same even if parallel mode

Reproducible project / your project

See my Dockerfile above.

Environment info

OS: Linux
Python version: 3.4
Sphinx version: 1.6.6

tk0miya commented 6 years ago

There are comment about duplication on domain. I think we have to check and warn duplications at here. https://github.com/sphinx-doc/sphinx/blob/020c51532bc70289926797e878856997415cd2f8/sphinx/domains/std.py#L549-L571

tk0miya commented 6 years ago

Fixed by #4882

tirkarthi commented 5 years ago

I just want to add that I tried this with CPython build and still the warning is not shown for duplicate labels.

Parallel build :

mkdir -p build
Building NEWS from Misc/NEWS.d with blurb
PATH=./venv/bin:$PATH sphinx-build -j auto -b html -d build/doctrees -D latex_elements.papersize=  . build/html
Running Sphinx v1.8.1
/Users/karthikeyansingaravelan/stuff/python/cpython/Doc/venv/lib/python3.6/site-packages/sphinx/application.py:927: RemovedInSphinx20Warning: app.add_description_unit() is now deprecated. Use app.add_object_type() instead.
  RemovedInSphinx20Warning)
making output directory...
building [mo]: targets for 0 po files that are out of date
building [html]: targets for 470 source files that are out of date
updating environment: 470 added, 0 changed, 0 removed
reading sources... [100%] tutorial/stdlib .. whatsnew/index
waiting for workers...
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
writing output... [100%] whatsnew/index
waiting for workers...
generating indices... genindex py-modindex
writing additional pages... download index search opensearch
copying images... [100%] using/win_installer.png
copying downloadable files... [100%] includes/tzinfo_examples.py
copying static files... done
copying extra files... done
dumping search index in English (code: en) ... done
dumping object inventory... done
build succeeded.

The HTML pages are in build/html.

Build finished. The HTML pages are in build/html.
make html  216.09s user 6.84s system 228% cpu 1:37.64 total

Non-parallel build : (duplicate label warning in example.rst is not shown for the parallel build)

mkdir -p build
Building NEWS from Misc/NEWS.d with blurb
PATH=./venv/bin:$PATH sphinx-build -b html -d build/doctrees -D latex_elements.papersize=  . build/html
Running Sphinx v1.8.1
/Users/karthikeyansingaravelan/stuff/python/cpython/Doc/venv/lib/python3.6/site-packages/sphinx/application.py:927: RemovedInSphinx20Warning: app.add_description_unit() is now deprecated. Use app.add_object_type() instead.
  RemovedInSphinx20Warning)
making output directory...
building [mo]: targets for 0 po files that are out of date
building [html]: targets for 470 source files that are out of date
updating environment: 470 added, 0 changed, 0 removed
reading sources... [100%] whatsnew/index
/Users/karthikeyansingaravelan/stuff/python/cpython/Doc/distutils/examples.rst:5: WARNING: duplicate label examples, other instance in /Users/karthikeyansingaravelan/stuff/python/cpython/Doc/c-api/typeobj.rst
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
writing output... [100%] whatsnew/index
generating indices... genindex py-modindex
writing additional pages... download index search opensearch
copying images... [100%] using/win_installer.png
copying downloadable files... [100%] includes/tzinfo_examples.py
copying static files... done
copying extra files... done
dumping search index in English (code: en) ... done
dumping object inventory... done
build succeeded, 1 warning.

The HTML pages are in build/html.

Build finished. The HTML pages are in build/html.
make html  162.59s user 2.51s system 95% cpu 2:53.40 total

Thanks

marlamb commented 2 weeks ago

@tk0miya as this issue is still open I comment here instead of opening a new issue. With sphinx version 7.3.7 we can again reproduce the described behavior. Is this a known limitation (given that closed intermediately) or a recent regression?

sphinx-doc / sphinx