[Documentation] Failing ReadTheDocs builds

robertodr commented 7 years ago

Builds on ReadTheDocs have been failing lately.

robertodr commented 7 years ago

Builds are still failing. @arnfinn @ilfreddy any chance you could have a look?

arnfinn commented 7 years ago

I can take a look. I have close to none experience in readthedocs builds, so don't expect too much:)

robertodr commented 7 years ago

Cool, don't hesitate to ask. You are now both admins for the ReadTheDocs page.

arnfinn commented 7 years ago

A couple of notes

commit 12210404 ok running Sphinx v1.3.5
commit d8adfbb4 unknown
commit 303dd253 failed running Sphinx v1.5.3

Thus, it is either commit d8adfbb4 or 303dd253, or going from Sphinx v1.3.5 to v1.5.3 that is causing the failed builds (probably the latter). Can probably find out either by:

replace sphinx with sphinx==1.3.5 in doc/requirements.txt and see if the build is OK, or
tag commit 12210404, build it and see if it fails

What do you think @robertodr? Is there a way of (easily) testing it locally?

Part of output from OK build:

Running Sphinx v1.3.5
...
...
...
Running dot...
lookup cache used 5495/65536 hits=56643 misses=5848
finished...
building [mo]: targets for 0 po files that are out of date
building [readthedocssinglehtmllocalmedia]: all documents
updating environment: 23 added, 0 changed, 0 removed
reading sources... [  4%] code-reference/cavities
reading sources... [  8%] code-reference/classes-and-functions
reading sources... [ 13%] code-reference/greens-functions
reading sources... [ 17%] code-reference/helper-classes
reading sources... [ 21%] code-reference/namespaces
reading sources... [ 26%] code-reference/solvers
reading sources... [ 30%] index
...

vs. failed build:

Running Sphinx v1.5.3
...
...
...
Running dot...
lookup cache used 5777/65536 hits=82941 misses=6132
finished...
building [mo]: targets for 0 po files that are out of date
building [readthedocssinglehtmllocalmedia]: all documents
updating environment: 23 added, 0 changed, 0 removed
reading sources... [  4%] code-reference/cavities
Killed

robertodr commented 7 years ago

Great work @arnfinn! You can test locally by doing make doc in the build directory. Look carefully in the ouput of the setup script to check that the dependencies for the doc target are all installed on you system. We can force usage of Sphinx v1.3.5, but it's better if we manage to make it work under the default Sphinx version of ReadTheDocs

bast commented 7 years ago

Good catch!

arnfinn commented 7 years ago

@robertodr: I am not able to build the docs locally.

$ make doc
make: *** No rule to make target `doc'.  Stop.

Sorry for my stupidity, but are there some other way I can do it?

robertodr commented 7 years ago

can you post the output of ./setup.py I suspect there's one or two missing dependencies.

arnfinn commented 7 years ago

Here you go (from Stallo):

FC=gfortran CC=gcc CXX=g++ cmake -DEXTRA_FCFLAGS="''" -DEXTRA_CFLAGS="''" -DEXTRA_CXXFLAGS="''" -DCMAKE_INSTALL_BINDIR=bin -DCMAKE_INSTALL_LIBDIR=lib -DCMAKE_INSTALL_INCLUDEDIR=include -DCMAKE_INSTALL_DATADIR=share -DUSE_CCACHE=ON -DPREPROCESSOR_DEFINITIONS="''" -DENABLE_CODE_COVERAGE=False -DENABLE_64BIT_INTEGERS=False -DENABLE_OPENMP=False -DPYTHON_INTERPRETER="''" -DENABLE_FORTRAN_API=ON -DBOOST_INCLUDEDIR="''" -DBOOST_LIBRARYDIR="''" -DFORCE_CUSTOM_BOOST=OFF -DBOOST_MINIMUM_REQUIRED="1.54.0" -DBOOST_COMPONENTS_REQUIRED="" -DSTATIC_LIBRARY_ONLY=False -DEIGEN3_ROOT="''" -DCMAKE_BUILD_TYPE=debug -G "Unix Makefiles" /home/arnfinn/repo/arnfinn/pcmsolver

-- The C compiler identification is GNU 5.4.0
-- The CXX compiler identification is GNU 5.4.0
-- Check for working C compiler: /global/hds/software/cpu/eb3/GCCcore/5.4.0/bin/gcc
-- Check for working C compiler: /global/hds/software/cpu/eb3/GCCcore/5.4.0/bin/gcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /global/hds/software/cpu/eb3/GCCcore/5.4.0/bin/g++
-- Check for working CXX compiler: /global/hds/software/cpu/eb3/GCCcore/5.4.0/bin/g++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- The Fortran compiler identification is GNU 5.4.0
-- Check for working Fortran compiler: /global/hds/software/cpu/eb3/GCCcore/5.4.0/bin/gfortran
-- Check for working Fortran compiler: /global/hds/software/cpu/eb3/GCCcore/5.4.0/bin/gfortran  -- works
-- Detecting Fortran compiler ABI info
-- Detecting Fortran compiler ABI info - done
-- Checking whether /global/hds/software/cpu/eb3/GCCcore/5.4.0/bin/gfortran supports Fortran 90
-- Checking whether /global/hds/software/cpu/eb3/GCCcore/5.4.0/bin/gfortran supports Fortran 90 -- yes
-- Detecting Fortran/C Interface
-- Detecting Fortran/C Interface - Found GLOBAL and MODULE mangling
-- Verifying Fortran/CXX Compiler Compatibility
-- Verifying Fortran/CXX Compiler Compatibility - Success
-- Performing Test _HAS_CXX11_FLAG
-- Performing Test _HAS_CXX11_FLAG - Success
-- C++11 "__func__" feature works
-- C++11 "auto" feature works
-- C++11 "auto_ret_type" feature works
-- C++11 "class_override_final" feature works
-- C++11 "constexpr" feature works
-- C++11 "cstdint" feature works
-- C++11 "decltype" feature works
-- C++11 "initializer_list" feature works
-- C++11 "lambda" feature works
-- C++11 "long_long" feature works
-- C++11 "nullptr" feature works
-- C++11 "regex" feature works
-- C++11 "rvalue-references" feature works
-- C++11 "sizeof_member" feature works
-- C++11 "static_assert" feature works
-- C++11 "variadic_templates" feature works
-- C++11 "noexcept" feature works
-- C++11 "noreturn" feature works
-- ccache not available
-- Found PythonInterp: /global/apps/Python/2.7.3/bin/python (found version "2.7.3")
-- Boost version: 1.61.0
-- PCMSolver v1.1.10
-- Eigen 3.3.2 is located here: /home/arnfinn/repo/arnfinn/pcmsolver/external/eigen3/include/eigen3
-- Found ZLIB: /usr/lib64/libz.so (found version "1.2.3")
-- Found Git: /usr/bin/git (found version "1.7.1")
-- Could NOT find Doxygen (missing:  DOXYGEN_EXECUTABLE)
-- Could NOT find Sphinx (missing:  SPHINX_EXECUTABLE)
-- Found Perl: /usr/bin/perl (found version "5.10.1")
-- Found PY_yaml: /global/apps/Python/2.7.3/lib/python2.7/site-packages/yaml
-- Could NOT find PY_breathe (missing:  PY_BREATHE)
-- Found PY_matplotlib: /global/apps/Python/2.7.3/lib/python2.7/site-packages/matplotlib
-- Performing Test COMPILER_HAS_HIDDEN_VISIBILITY
-- Performing Test COMPILER_HAS_HIDDEN_VISIBILITY - Success
-- Performing Test COMPILER_HAS_HIDDEN_INLINE_VISIBILITY
-- Performing Test COMPILER_HAS_HIDDEN_INLINE_VISIBILITY - Success
-- Performing Test COMPILER_HAS_DEPRECATED_ATTR
-- Performing Test COMPILER_HAS_DEPRECATED_ATTR - Success
-- System                : Linux
-- Processor type        : x86_64
-- C++ compiler flags    :   -std=c++11 -O0 -g3 -DDEBUG -Wall -Wextra -Winit-self -Woverloaded-virtual -Wuninitialized -Wmissing-declarations -Wwrite-strings -Wno-sign-compare
-- C compiler flags      :   -std=c99 -DRESTRICT=restrict -DFUNDERSCORE=1 -fPIC -O0 -g3 -DDEBUG -Wall -Wextra -Winit-self -Wuninitialized -Wmissing-declarations -Wwrite-strings -Wno-sign-compare
-- Fortran compiler flags:   -fimplicit-none -fautomatic -fmax-errors=5 -O0 -g -fbacktrace -Wall
-- Definitions           : HAS_CXX11;HAS_CXX11_FUNC;HAS_CXX11_AUTO;HAS_CXX11_AUTO_RET_TYPE;HAS_CXX11_CLASS_OVERRIDE;HAS_CXX11_CONSTEXPR;HAS_CXX11_CSTDINT_H;HAS_CXX11_DECLTYPE;HAS_CXX11_INITIALIZER_LIST;HAS_CXX11_LAMBDA;HAS_CXX11_LONG_LONG;HAS_CXX11_NULLPTR;HAS_CXX11_LIB_REGEX;HAS_CXX11_RVALUE_REFERENCES;HAS_CXX11_SIZEOF_MEMBER;HAS_CXX11_STATIC_ASSERT;HAS_CXX11_VARIADIC_TEMPLATES;HAS_CXX11_NOEXCEPT;HAS_CXX11_NORETURN;ENABLE_TIMER;PCMSolver_EXPORTS;TAYLOR_CXXIO
-- Configuring done
-- Generating done
-- Build files have been written to: /home/arnfinn/repo/arnfinn/pcmsolver/build-tmp

   configure step is done
   now you need to compile the sources:
   $ cd build-tmp
   $ make

robertodr commented 7 years ago

These are the offending lines:

-- Could NOT find Doxygen (missing:  DOXYGEN_EXECUTABLE)
-- Could NOT find Sphinx (missing:  SPHINX_EXECUTABLE)
-- Found Perl: /usr/bin/perl (found version "5.10.1")
-- Found PY_yaml: /global/apps/Python/2.7.3/lib/python2.7/site-packages/yaml
-- Could NOT find PY_breathe (missing:  PY_BREATHE)
-- Found PY_matplotlib: /global/apps/Python/2.7.3/lib/python2.7/site-packages/matplotlib

Sphind and Breathe can be installed using virtualenv. Note to self: document this :cry:

arnfinn commented 7 years ago

It might be related to bug 3709 in Sphinx, which is not a bug in Sphinx but in Breathe. It has been fixed in Breathe pull request 334, but probably not in an official release yet.

It is working with Sphinx==1.5.6. Maybe do that for now?

EDIT: That was probably another bug, when I tried to compile locally. RTD is using Sphinx v1.5.3...

robertodr commented 7 years ago

I am confused, what's the workaround that works? Sphinx==1.5.6? It is possible to point Pip to use the latest commit on Breathe master branch by adding this to the requirements.txt

-e git+https://github.com/michaeljones/breathe.git@03ea574ef6034019c2e928c5733b24e64ec8b651#egg=breathe

arnfinn commented 7 years ago

Sorry for the confusion @robertodr. My fault:)

The build failed on Stallo, and it failed because of this bug in Breathe using Sphinx 1.6.x. So I thought this was the reason why it failed on RTD. But then I realized that RTD used Sphinx 1.5.x, so it has probably nothing to do with the bug in Breathe (the build is not failing on Stallo with the same version of Sphinx as used by RTD).

I will continue looking for the reason why it is failing on RTD.

robertodr commented 7 years ago

OK, I understand now. Good luck with the bug hunt. If it doesn't bear any fruit we can do sphinx==1.3.5 and see if that works on RTD.

robertodr commented 7 years ago

Hi @arnfinn, how's debugging of this going?

arnfinn commented 7 years ago

Up and down:)

I am able to build it after a wipe (cleaning out the build checkouts and environment), and maybe one more time. After that, it always fails.

Not sure how to proceed, but we can talk about it next week?

robertodr commented 7 years ago

Annoying that you need to go in manually after a couple of times... Maybe it's possible to wipe intermediate files in the conf.py script? We can surely talk about it next week. Do they have a forum where we can ask for help?

robertodr commented 7 years ago

Status so far: it seems using breathe syntax in .rst files makes RTD crash.

robertodr commented 7 years ago

@arnfinn news on this?

arnfinn commented 7 years ago

Will take a look next week. I am working at the university monday to wednesday

arnfinn commented 7 years ago

Probably nothing, but writing it down so I will not forget:

I thought we removed all the doxygenclass: (breathe syntax?) in the rtd_testing branch, but it seems like we forgot the one in users/interfacing.rst. The build was killed in this file at one time, but when it passed (in the next push) the text was not show. Instead a warning was shown on the documentation page:

doxygenclass: Cannot find class “Meddle” in doxygen xml output for project “PCMSolver” from directory: xml

These warnings also show up in the release/1.Y documentation (when the build passes), see for instance this one..

In other words, even if the build passes the documentation is not complete.

arnfinn commented 7 years ago

Again, probably nothing, but writing it down so I will not forget.

For failed build, this is seen:

Generating XML output for file cavities.txt
Generating XML output for file classes-and-functions.txt
Generating XML output for file greens-functions.txt
Generating XML output for file helper-classes.txt
Generating XML output for file namespaces.txt
Generating XML output for file solvers.txt
Generating XML output for file index.txt
Generating XML output for file cmake-usage.txt
Generating XML output for file coding-standards.txt
Generating XML output for file documentation.txt
Generating XML output for file general-structure.txt
Generating XML output for file maintenance.txt
Generating XML output for file programmers-manual.txt
Generating XML output for file testing.txt
Generating XML output for file timer-class.txt
Generating XML output for file publications.txt
Generating XML output for file building.txt
Generating XML output for file C-example.txt
Generating XML output for file fortran-example.txt
Generating XML output for file input.txt
Generating XML output for file interfacing.txt
Generating XML output for file users-manual.txt
Generating XML output for file zreferences.txt
Generating XML output for file doctools.js
Generating XML output for file jquery-1.11.1.js
Generating XML output for file jquery.js
Generating XML output for file modernizr.min.js
Generating XML output for file theme.js
Generating XML output for file searchtools.js
Generating XML output for file underscore-1.3.1.js
Generating XML output for file underscore.js
Generating XML output for file websupport.js

robertodr commented 6 years ago

Solved with #102. Thanks @arnfinn :tada: Closing

PCMSolver / pcmsolver

[Documentation] Failing ReadTheDocs builds #61

A couple of notes