doc: switch back to libclang to avoid hardcoding LLVM version

[bgilbert]

The libclang package has an embedded copy of libclang, which allows us to avoid requiring the system to have a particular clang version. This didn't work before because the lib_search_dirs logic in conf.py was clobbering a working path with a broken one. That logic is no longer needed, so just remove it.

Tested on Fedora 38 and Ubuntu 22.04, but not specifically tested with Read the Docs.

[jcupitt]

Oh nice. Let's try this!

[jcupitt]

Sadly this breaks on readthedocs with what looks like a clang14 / clang16 mismatch error:

clang.cindex.LibclangError: /usr/lib/llvm-14/lib/libclang.so.1: undefined symbol: clang_CXXMethod_isDeleted. 
Please ensure that your python bindings are compatible with your libclang.so version.

The build error report says:

>>> python -m sphinx -T -E -b html -d _build/doctrees -D language=en . $READTHEDOCS_OUTPUT/html
Running Sphinx v7.2.6
making output directory... done
WARNING: html_static_path entry '_static' does not exist
building [mo]: targets for 0 po files that are out of date
writing output... 
building [html]: targets for 7 source files that are out of date
updating environment: [new config] 7 added, 0 changed, 0 removed
reading sources... [ 14%] api

Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/clang/cindex.py", line 4255, in register_function
    func = getattr(lib, item[0])
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/docs/.asdf/installs/python/3.11.4/lib/python3.11/ctypes/__init__.py", line 389, in __getattr__
    func = self.__getitem__(name)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/docs/.asdf/installs/python/3.11.4/lib/python3.11/ctypes/__init__.py", line 394, in __getitem__
    func = self._FuncPtr((name_or_ordinal, self))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: /usr/lib/llvm-14/lib/libclang.so.1: undefined symbol: clang_CXXMethod_isDeleted

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/sphinx/cmd/build.py", line 298, in build_main
    app.build(args.force_all, args.filenames)
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/sphinx/application.py", line 355, in build
    self.builder.build_update()
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/sphinx/builders/__init__.py", line 293, in build_update
    self.build(to_build,
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/sphinx/builders/__init__.py", line 313, in build
    updated_docnames = set(self.read())
                           ^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/sphinx/builders/__init__.py", line 420, in read
    self._read_serial(docnames)
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/sphinx/builders/__init__.py", line 441, in _read_serial
    self.read_doc(docname)
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/sphinx/builders/__init__.py", line 498, in read_doc
    publisher.publish()
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/docutils/core.py", line 217, in publish
    self.document = self.reader.read(self.source, self.parser,
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/sphinx/io.py", line 105, in read
    self.parse()
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/docutils/readers/__init__.py", line 78, in parse
    self.parser.parse(self.input, document)
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/sphinx/parsers.py", line 81, in parse
    self.statemachine.run(inputlines, document, inliner=self.inliner)
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/docutils/parsers/rst/states.py", line 170, in run
    results = StateMachineWS.run(self, input_lines, input_offset,
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/docutils/statemachine.py", line 240, in run
    context, next_state, result = self.check_line(
                                  ^^^^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/docutils/statemachine.py", line 452, in check_line
    return method(match, context, next_state)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/docutils/parsers/rst/states.py", line 2779, in underline
    self.section(title, source, style, lineno - 1, messages)
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/docutils/parsers/rst/states.py", line 327, in section
    self.new_subsection(title, lineno, messages)
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/docutils/parsers/rst/states.py", line 393, in new_subsection
    newabsoffset = self.nested_parse(
                   ^^^^^^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/docutils/parsers/rst/states.py", line 281, in nested_parse
    state_machine.run(block, input_offset, memo=self.memo,
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/docutils/parsers/rst/states.py", line 196, in run
    results = StateMachineWS.run(self, input_lines, input_offset)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/docutils/statemachine.py", line 240, in run
    context, next_state, result = self.check_line(
                                  ^^^^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/docutils/statemachine.py", line 452, in check_line
    return method(match, context, next_state)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/docutils/parsers/rst/states.py", line 2352, in explicit_markup
    nodelist, blank_finish = self.explicit_construct(match)
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/docutils/parsers/rst/states.py", line 2364, in explicit_construct
    return method(self, expmatch)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/docutils/parsers/rst/states.py", line 2101, in directive
    return self.run_directive(
           ^^^^^^^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/docutils/parsers/rst/states.py", line 2151, in run_directive
    result = directive_instance.run()
             ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/sphinx/domains/__init__.py", line 289, in run
    return super().run()
           ^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/hawkmoth/__init__.py", line 152, in run
    self.__parse(filename)
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/hawkmoth/__init__.py", line 80, in __parse
    docstrings, errors = parse(filename, domain=self._domain,
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/hawkmoth/parser.py", line 755, in parse
    index = Index.create()
            ^^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/clang/cindex.py", line 2875, in create
    return Index(conf.lib.clang_createIndex(excludeDecls, 0))
                 ^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/clang/cindex.py", line 212, in __get__
    value = self.wrapped(instance)
            ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/clang/cindex.py", line 4336, in lib
    register_functions(lib, not Config.compatibility_check)
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/clang/cindex.py", line 4283, in register_functions
    register(f)
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/clang/cindex.py", line 4280, in register
    return register_function(lib, item, ignore_errors)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/clang/cindex.py", line 4261, in register_function
    raise LibclangError(msg)
clang.cindex.LibclangError: /usr/lib/llvm-14/lib/libclang.so.1: undefined symbol: clang_CXXMethod_isDeleted. Please ensure that your python bindings are compatible with your libclang.so version.

Exception occurred:
  File "/home/docs/checkouts/readthedocs.org/user_builds/libdicom/envs/latest/lib/python3.11/site-packages/clang/cindex.py", line 4261, in register_function
    raise LibclangError(msg)
clang.cindex.LibclangError: /usr/lib/llvm-14/lib/libclang.so.1: undefined symbol: clang_CXXMethod_isDeleted. Please ensure that your python bindings are compatible with your libclang.so version.
The full traceback has been saved in /tmp/sphinx-err-19649r46.log, if you want to report the issue to the developers.
Please also report this if it was a user error, so that a better error message can be provided next time.

[jcupitt]

Maybe requirement.txt needs libclang <= 14?

[jcupitt]

Recent readthedocs builds are here: https://readthedocs.org/projects/libdicom/builds/

[bgilbert]

Oof. I wonder why it works on vanilla 22.04 and fails on ReadTheDocs. Maybe we can use a separate requirements.txt for ReadTheDocs? I don't think it's a good idea to hardcode an LLVM 14 requirement in the Meson build, because the docs need to be buildable on other distros as well. E.g., Hawkmoth is in the process of being packaged for Fedora, and once it lands, the Fedora libdicom package will rebuild the docs from source.

[jcupitt]

Yes, it's a bit annoying, I spent a morning getting readthedocs working :( I wish it was simpler.

I'll revert this commit for now to get the docs building again.

[jcupitt]

They have their build images here, fwiw: https://github.com/readthedocs/readthedocs-docker-images

[bgilbert]

Second attempt in #74.