Autodoxygen source option not known

[scopatz]

As of the current master v1.2 (b30dee9b4d83), I am now seeing a lof of breakage with the autodoxygenindex directive:

/home/scopatz/pyne/docs/cppapi/rxname.rst:4: ERROR: Error in "autodoxygenindex" directive:
unknown option: "source".

I don't know when this snuck in, but now breathe doesn't work at all with automatic tools.

[michaeljones]

Hi scopatz,

Thank you reporting this. Sorry that it seem to be broken. I'm just getting my set up back together and I should be able to check this out soon. I am sorry for the trouble. I'm not sure what would have gone wrong.

Michael

[scopatz]

Thanks for looking into it @michaeljones

[sushant-hiray]

@michaeljones I'm also facing a similar issue.

/home/sushant/code/csympy/doc/src/basic.rst:4: ERROR: Unknown directive type "autodoxygen".

.. autodoxygen:: basic.h
      :source: csympy

Is this fixed yet? I installed breathe via pip

[michaeljones]

Hi,

This hasn't been fixed yet but I am finally in place to start looking into it. I'll let you know in the next few days how I get on.

Sorry for the hold up, I've been moving countries and travelling a little in between so it has been a while since I've had a decent development set up.

Michael

[michaeljones]

Hi

So @sushant-hiray, your issue is because the documentation is wrong. My bad. The rst directive is actually called autodoxygenindex rather than autodoxygen. I will update the docs now. If you look at the source of the docs it is stating one and then using the other. Sorry!

@scopatz are you still experiencing a problem with this? The latest master is working fine for me if I do a build of the docs and go to the example page. The commit that you have referenced (b30dee9b4d83) is from before some of the major work on the autodoxygenindex directive and is long before the current 1.2 tag. Could you take another look? Sorry for the confusion.

Thanks, Michael

[sushant-hiray]

Hi, @michaeljones thanks for the response. I'll look into it and post back if there are any further issues.

Thanks, Sushant

[scopatz]

Thanks @michaeljones, you are correct that I mistakenly seemed to be on an older version of the code. Thanks! Feel free to close if you wish. I am now experiencing an issue where multiple uses of autodoxygenindex will all appear with the same value, even if they are pointed at different files. You can see this if you looks at https://github.com/pyne/pyne. I can open up another issue once I track this down.

[michaeljones]

Ah, thank you for letting me know. Shame about the bug with the auto stuff. I'll try to have a look at reproducing it and seeing what it going on. Let me know if you find anything.

Cheers, Michael

[scopatz]

Will do :)

[michaeljones]

Hey,

So, I haven't tested anything yet, but it struck me last night that you might want a different source for each autooxygenindex directive usage. Currently they all refer to pyne and I think that under the hood, Breathe collects together all the source files that use the same source and runs doxygen on the whole lot. Each autodoxygenindex would then bring in everything together at each place. I might be wrong as I haven't looked at the output yet but you might want to try:

.. autodoxygenindex:: material.h
   :source: material

then

.. autodoxygenindex:: data.h
   :source: data

Unfortunately this means that in your Breathe config you'll need multiple entries in the breathe_source_projects dictionary; all pointing to the same folder.

This seems a little irrational as a design but I think my plan was that if we introduced more auto directives. Like for class and function and other things. I didn't want each one creating a new folder and running doxygen under the hood if they all pointed to the same set of sources files and they could be collected together and ran once. Collecting them together also helps with the cross linking I should think.

I imagine what you really want in this case is some kind of autodoxygenfile directive where everything would be handled automatically but you'd only get the file contents. This might not be too hard to put together. The approach I've suggested is a hack to pretend each file is its own project so that we can use the index approach which is really designed to dump the whole contents of the project's code base.

Does that make sense? Bit of a ramble, I'm afraid.

Michael

[sushant-hiray]

Hey, I was facing the same problem which @scopatz was facing. I shifted all the autodoxygenindex from a single source to different source for each file.

Still there hasn't been any change. So all the files contain the same html output. You can look the pull here. Could you test the autodoxygenindex and look into it.

I agree that a much better option, we are trying to get is autodoxygenfile kind of thingy, which could automate the entire process.

Regards, Sushant

[michaeljones]

Thank you for the input. I'll try to have a look into it.

Michael

[scopatz]

Hi @michaeljones! I suspected that autodoxygenfile and other directives would be the right course of action. In the short term, I'll try try the different sources trick and let you know if that works.

[michaeljones]

Thanks, though it seems that Sushant has tried that and not been very lucky. I'll try to set up an example tomorrow and look into it more closely. The code might be doing the grouping in the background in another way and getting confused. I'd hope my suggestion would work and I'll try to fix it if it doesn't, but then, yeah, it makes sense to have a look at an autodoxygenfile directive for you guys.

Cheers, Michael

[scopatz]

Unfortuanetly, I can confirm now that this did not work. It looks like there are no quick shortcuts.

[michaeljones]

Hi @scopatz @sushant-hiray,

I've just pushed a branch called auto which I am hoping addresses this problem. It was a small change and in the autodoxygenindex code path so I'm hoping it doesn't cause any issues else where.

If you guys could check it out and let me know if it works for you (I've tested with a simple set up) then I'll merge and release something.

Cheers for all the help, Michael

[scopatz]

Hello @michaeljones, this works like a dream with the different source trick described above! It has the same problems as before if you have the same source. I am using the following in my conf.py that you may want to refernce in the docs as an example:

# C++ Autodocumentation Flags
breathe_projects = {"pyne": None,}
breathe_default_project = 'pyne'
breathe_domain_by_extension = {"h": "cpp",}
breathe_projects_source = {"pyne": '../cpp',}

for p in os.listdir(breathe_projects_source['pyne']):
    p, _ = os.path.splitext(p)
    breathe_projects['pyne_' + p] = breathe_projects['pyne']
    breathe_projects_source['pyne_' + p] = breathe_projects_source['pyne']

This can be used with:

.. autodoxygenindex:: data.h
    :source: pyne_data

Obviously people might need to replace 'pyne' and their own files, but it is a pretty clean strategy.

[scopatz]

Thanks again for this by the way! This is right in time for a PyNE release :).

[michaeljones]

Cheers for the update. Much appreciated and I'm glad it is working.

I've actually just pushed an autofile branch with an autodoxygenfile directive. There is an example in the docs on that branch. Mostly looks like:

.. autodoxygenfile:: data.h
   :source: pyne

ie. You don't have to provide a different source for every instance. I would welcome feedback if you get the chance to test it out.

Thanks again for persevering with Breathe! Michael

[scopatz]

Hi @michaeljones! Thanks, I tried this out and it almost works. For the (last?) file everything seems great - here data.h. However for all other webpages, the html contains warnings of the following form:

Warning doxygenfile: Cannot find file “material.h” in doxygen xml output for project “pyne” from directory: /home/scopatz/pyne/docs/_build/breathe/doxygen/pyne/xml

It seems like when multiple files are used the pyne xml file is being deleted or over written.

To be clear, I have the following setup:

material.rst

Materials -- ``libpyne_material``
=================================

.. autodoxygenfile:: material.h
    :source: pyne

data.rst

Nuclear Data -- ``libpyne_data``
=====================================

.. autodoxygenfile:: data.h
    :source: pyne

[michaeljones]

Thanks for testing it out and sorry for not catching that. I didn't realise that the Transform functionality was run separately on each file by Sphinx. My "more complex" test was two directives in the same file. I'll try to find a way to make it work across all the files.

I'll let you know if I get anywhere. Thank you for your time.

Michael

[scopatz]

Thanks for persevering!

[michaeljones]

I'm afraid I cannot see a clear way forward with the current Sphinx API and events system. I have registered an issue with the Sphinx devs to see if I can get any help:

https://bitbucket.org/birkenfeld/sphinx/issue/1440/two-pass-doctree-transforms

I'll post on the sphinx-dev mailing list as well and we'll see how we get on.

Cheers, Michael

[scopatz]

Thanks! I upvoted the issue!

[michaeljones]

Hey,

I've not really heard much feedback from the Sphinx guys and looking at the Sphinx source code I really can't see an appropriate place to put the hooks we need. Perhaps someone with more experience will be able to tell but for the moment it looks a little bleak.

An alternative is that we provide more information up front; instead of Breathe having to discover the source files required for doxygen from the rst documents, we could provide a configuration variable that lists all the required source files by project. Then you'd have something like:

breathe_project_folders = {
    "pyne" : "../cpp/pyne"
    }

breathe_project_source = {
    "pyne" : [ "data.h", "material.h" ]
    }

Or something similar. Then breathe can know what files to process with doxygen up front and we get around this issue.

Is that tolerable? You still have Breathe looking after the doxygen call and file management.

Cheers, Michael

[scopatz]

Hi @michaeljones. That is totally acceptable to me! Sorry for not getting back for so long. (PyCon happened ~_~.) I think that API that you suggested looks great.

[michaeljones]

Hi @scopatz,

Cheers for getting back to me. Hope you enjoyed PyCon! I'm afraid I've been fairly busy as well but I've just pushed some changes to the autofile branch which hopefully will work well with this new config based approach. In the end, I've gone with a config value of:

breathe_projects_source = {
    "pyne" : ( "../cpp/pyne", [ "data.h", "material.h" ] )
    }

I've updated the documentation on that branch to reflect the changes.

I would greatly appreciate it if you have a chance to give it a go. I've tried to test the various combinations but there is nothing like a proper project to give it a go with.

Cheers, Michael

[michaeljones]

Hi,

The docs are here:

• http://breathe.readthedocs.org/en/latest/directives.html#autodoxygenfile
• http://breathe.readthedocs.org/en/latest/directives.html#confval-breathe_projects_source

I'm going to close this ticket but I'm very happy to open up more for other more specific issues and additional features.

Cheers! Michael

[scopatz]

Thanks @michaeljones!

[leimao]

Could you please verify this is still working? It seems that it is broken at my side.

/home/leimao/GitHub/Sphinx-CPP-TriangleLib/docs/source/api.rst:9: WARNING: Error in "autodoxygenfile" directive:
unknown option: "source".

.. autodoxygenfile:: triangle.h
   :source: auto

[SooluThomas]

Was this fixed? I'm getting the same error:

ERROR: Error in "autodoxygenfile" directive:
unknown option: "source".

[vermeeren]

@leimao @SooluThomas The option is called :project: nowadays, not :source:. This has been the case for years, I don't personally remember a time where it was :source:. #551 updated some docs for this in 2020.

Edit: see also for example the doc https://breathe.readthedocs.io/en/latest/autofile.html#autodoxygenfile-example