0.36.0 release not compatible with sphinx autoapi plugin

[rsokolewicz]

We use sphinx to create documentation for our quantify-scheduler and quantify-core packages. It looks like when we have qcodes==0.36.0 installed the building of the documentation fails.

Unfortunately, the stack trace is not informative in where and why it happens:

[AutoAPI] Reading files... [  1%] /home/docs/checkouts/readthedocs.org/user_builds/quantify-quantify-scheduler/checkouts/545/quantify_scheduler/gettables_profiled.py

Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/quantify-quantify-scheduler/envs/545/lib/python3.8/site-packages/sphinx/events.py", line 94, in emit
    results.append(listener.handler(self.app, *args))
  File "/home/docs/checkouts/readthedocs.org/user_builds/quantify-quantify-scheduler/envs/545/lib/python3.8/site-packages/autoapi/extension.py", line 164, in run_autoapi
    if sphinx_mapper_obj.load(
  File "/home/docs/checkouts/readthedocs.org/user_builds/quantify-quantify-scheduler/envs/545/lib/python3.8/site-packages/autoapi/mappers/python/mapper.py", line 306, in load
    data = self.read_file(path=path, dir_root=dir_root)
  File "/home/docs/checkouts/readthedocs.org/user_builds/quantify-quantify-scheduler/envs/545/lib/python3.8/site-packages/autoapi/mappers/python/mapper.py", line 323, in read_file
    parsed_data = Parser().parse_file(path)
  File "/home/docs/checkouts/readthedocs.org/user_builds/quantify-quantify-scheduler/envs/545/lib/python3.8/site-packages/autoapi/mappers/python/parser.py", line 41, in parse_file
    return self._parse_file(
  File "/home/docs/checkouts/readthedocs.org/user_builds/quantify-quantify-scheduler/envs/545/lib/python3.8/site-packages/autoapi/mappers/python/parser.py", line 37, in _parse_file
    node = astroid.builder.AstroidBuilder().file_build(file_path, module_name)
  File "/home/docs/checkouts/readthedocs.org/user_builds/quantify-quantify-scheduler/envs/545/lib/python3.8/site-packages/astroid/builder.py", line 135, in file_build
    return self._post_build(module, builder, encoding)
 ...
   File "/home/docs/checkouts/readthedocs.org/user_builds/quantify-quantify-scheduler/envs/545/lib/python3.8/site-packages/astroid/raw_building.py", line 82, in attach_const_node
    _attach_local_node(node, nodes.const_factory(value), name)
  File "/home/docs/checkouts/readthedocs.org/user_builds/quantify-quantify-scheduler/envs/545/lib/python3.8/site-packages/astroid/nodes/node_classes.py", line 5531, in const_factory
    instance.postinit(_create_dict_items(value, instance))
  File "/home/docs/checkouts/readthedocs.org/user_builds/quantify-quantify-scheduler/envs/545/lib/python3.8/site-packages/astroid/nodes/node_classes.py", line 5502, in _create_dict_items
    for key, value in values.items():
RuntimeError: dictionary changed size during iteration

just that it occurs right after autoapi is running.

After some trial and error I found out that if I do a pip install --no-deps qcodes<0.36.0 the build succeeds, and when I do a pip install --no-deps qcodes==0.36.0 it fails.

Unfortunately, I don't have a minimal working example other than pip installing our quantify package and building the documentation locally:

mkdir tmp
cd tmp
git clone --no-single-branch --depth 50 git@gitlab.com:quantify-os/quantify-scheduler.git .
git fetch origin --force --tags --prune --prune-tags --depth 50 merge-requests/545/head:external-545
git checkout --force a7cc1ff360a2da8bb64f44f405a5b7bc5f3389fc
git clean -d -f -f
conda create --name tmp python=3.8
conda activate tmp
pip install quantify-scheduler
pip install -r requirements_dev.txt
pip install --no-deps qcodes==0.36.0
cd docs
python -m sphinx -T -E -b html -d _build/doctrees -D language=en . _build/html

this will fail the build with the error I pasted above. After doing a pip install --no-deps qcodes<0.36.0 it succeeds.

[jenshnielsen]

Are you using the QCoDeS sphinx plugin ?

The only other thing that I can think of is the lazy loading that was added with https://github.com/QCoDeS/Qcodes/pull/4788

[rsokolewicz]

Hi @jenshnielsen, could you share a link to the plugin you mentioned? I'm not able to find it myself, unfortunately. I will investigate the lazy loading thing.

[rsokolewicz]

importing h5py directly solves the issue :smile:. For completeness, I added the line import h5py inside our conf.py file and now the documentation build succeeds. We can close this issue, but I'm still curious about the qcodes sphinx plugin/extension.

[jenshnielsen]

Great to hear that we found this issue. It would probably be good to report it over here https://github.com/scientific-python/lazy_loader/issues if we can find a way to reproduce it a bit simpler.

We are using this https://github.com/QCoDeS/Qcodes/blob/master/qcodes/sphinx_extensions/parse_parameter_attr.py extension to be able to document Parameters on instruments a bit better (instance attributes defined in __init__

[rsokolewicz]

Thanks :) created an issue over here https://github.com/scientific-python/lazy_loader/issues/37