AiiDA sphinx extension: expose_inputs / expose_outputs not working on RTD

[danieleongari]

In the alpha-1 release the documentation is failing and the autodocumented API is not working as expected: it is incomplete.

[ltalirz]

Is this still an issue?

[danieleongari]

Yes, the two problems were unrelated: now the badge says that "docs passing" but still the API is not showing inputs/outputs.

[ltalirz]

@greschd we have a very strange issue here:

In the documentation built on readthedocs.org, the autodoc of a number of WorkChains is missing the outputs: https://aiida-lsmo.readthedocs.io/en/latest/workflows.html#isotherm-work-chain (see code here: https://github.com/lsmo-epfl/aiida-lsmo/blob/develop/aiida_lsmo/workchains/isotherm.py#L244 )

When building the documentation locally, it works fine (all information is present).

The RTD build shows no errors (see https://readthedocs.org/projects/aiida-lsmo/builds/10029959/) and the build environment seems to be the same as my local environment:

• python3.6
• sphinx 2.2.1
• sphinxcontrib-contentui 0.2.4
• sphinxcontrib-details-directive 0.1.0

We have

• checked that we're on the same branch
• wiped the build env on rtd, to start from scratch without caches

We have run out of ideas what could be causing this - do you happen to have any?

[greschd]

Maybe the versions of aiida-zeopp installed are not the same?

[greschd]

Or could be because the reentry scan is missing. The "magic lines of code" for the RTD setup in conf.py has changed for 1.0, see for example https://github.com/greschd/aiida-optimize/blob/develop/doc/source/conf.py#L32

[greschd]

If that's the issue, it's weird that it doesn't complain about a missing entrypoint error though.

[ltalirz]

Thanks for the suggestions

Maybe the versions of aiida-zeopp installed are not the same?

I have the same locally. Any reason to single out aiida-zeopp or does your concern apply to all dependencies of aiida-lsmo?

Or could be because the reentry scan is missing. The "magic lines of code" for the RTD setup in conf.py has changed for 1.0, see for example https://github.com/greschd/aiida-optimize/blob/develop/doc/source/conf.py#L32 If that's the issue, it's weird that it doesn't complain about a missing entrypoint error though.

Yeah, we ran into this issue some time ago and then added the scan https://github.com/lsmo-epfl/aiida-lsmo/blob/fe211bdbbdd13e4edc63bbf8645b718cd4ee4c06/docs/source/conf.py#L25-L30

One difference I can see to your conf.py is the import order - I think yours is sensible. I'll give this a try on the off chance that it changes anything.

[ltalirz]

Further tests:

• as expected, import order did not change anything
• unable to reproduce locally, even with exactly the same packages (see requirements.txt scraped from build log)
• issue persists when disabling the sphinxcontrib-details-directive
• issue persists when downgrading to sphinx 1.x

Time to think...

[greschd]

I have the same locally. Any reason to single out aiida-zeopp or does your concern apply to all dependencies of aiida-lsmo?

Just because I noticed that it's exposing outputs from the ZeoppCalculation - so I was wondering if there might be something going wrong there.

Yeah, we ran into this issue some time ago and then added the scan

https://github.com/lsmo-epfl/aiida-lsmo/blob/fe211bdbbdd13e4edc63bbf8645b718cd4ee4c06/docs/source/conf.py#L25-L30

Ah, I hadn't seen that there.

At this point I would probably add some debug statements to https://github.com/aiidateam/aiida-core/blob/196a074c45774de2c4bde6dbebb8b952b695eb4a/aiida/sphinxext/process.py#L122 (branching off from the 1.0 release, to not change anything else) and try installing with that on RTD.

The first thing to figure out is whether the outputs is actually empty on RTD, or if it's just the sphinx plugin doing something fishy.

[greschd]

It seem suspicious at least that it happens for a workchain where outputs are exposed from a different package. It seems the same happens for the MultistageDdec as well, so it's probably not zeopp specific.

[greschd]

Maybe try also importing these in the conf.py. I don't know the specifics, but I think sphinx does some weird things in how exactly it loads modules.

As a complete aside, I noticed that just doing automodule or autoclass no longer triggers the AiiDA specific command. Do you know if that is on purpose? I think that worked at some point, but there could of course be value in having both available.

[ltalirz]

It seem suspicious at least that it happens for a workchain where outputs are exposed from a different package.

Very good observation - indeed this is the source of the problem. For the documentation to build on RTD, importing the python files must not require the DB to be present. Spinx will throw errors when this happens for the source code of the package to be documented (here aiida-lsmo), but it seems that the same errors are suppressed if they happen when importing the source code of other packages (I suspect aiida-raspa is one of the perpetrators).

I'm now able to reproduce the problem locally with export READTHEDOCS=True and making sure to set the AIIDAPATH to a path without a valid configuration.

As a complete aside, I noticed that just doing automodule or autoclass no longer triggers the AiiDA specific command. Do you know if that is on purpose? I think that worked at some point, but there could of course be value in having both available.

Since I made the changes, I might have screwed that up unintentionally - was certainly not on purpose. I'll have a look into fixing this, thanks for reporting.

[ltalirz]

Just for the record, these were the culprits: https://github.com/yakutovicha/aiida-raspa/blob/10137785018838f9ec6c708e5d22cd4a8c648c0a/aiida_raspa/workchains/aiida_base_restart.py#L95-L98

Will open an issue on aiida-core - we need to make this easier to debug

[greschd]

Since I made the changes, I might have screwed that up unintentionally - was certainly not on purpose. I'll have a look into fixing this, thanks for reporting.

So that we don't forget this, I made an issue for this: https://github.com/aiidateam/aiida-core/issues/3689

[danieleongari]

Now working well!