Since Doxygen 1.9.7, file / namespace docs are useless when groups are used

[danblae]

Hello,

I got the following error (I adjusted the file paths since do not want to share details for the project and for the sake of readability).

Any recommendations what to do or how to get a more detailed output which of the over 100 XML files it has a problem with?

(I tried `doxygen.py doxyfile-mcss --debug and that is the output)

[build] Traceback (most recent call last):
[build]   File "doxygen.py", line 4015, in <module>
[proc] The command:cmake.EXE --build Debug --target all exited with code: 1
[build]     run(state, templates=os.path.abspath(args.templates), wildcard=args.wildcard, index_pages=args.index_pages, search_merge_subtrees=not args.search_no_subtree_merging, search_add_lookahead_barriers=not args.search_no_lookahead_barriers, search_merge_prefixes=not args.search_no_prefix_merging)
[build]   File "doxygen.py", line 3872, in run
[build]     parsed = parse_xml(state, file)
[build]   File "doxygen.py", line 2941, in parse_xml
[build]     enum = parse_enum(state, memberdef)
[build]   File "doxygen.py", line 1853, in parse_enum
[build]     assert element.tag == 'memberdef' and element.attrib['kind'] == 'enum'
[build] AssertionError

[marzer]

@danblae Which version of doxygen are you using? I saw a similar issue last week with 1.9.7 and have a workaround, but until now I thought it was just a side-effect of some niche thing I was doing.

[danblae]

@marzer I am using 1.9.7. Have you shared that workaround somewhere already? If not would you be so kind to share it?

[marzer]

Only as a part of poxy, not as any user-accessible doxygen-side workaround. I realize that's not overly helpful for you in your specific situation; I'll first confirm that this is indeed the same bug, then I'll get some information for @mosra to be able to fix it m.css-side (IIRC it was a fairly simple XML goof that needed special-casing).

[marzer]

Ok, so it seems it's not the exact same bug, but I suspect it has the same underyling cause. @danblae, do you mind invoking doxgen as 'xml-only' (i.e. disable HTML output, enable XML output, and invoke Doxygen directly), zipping the output, and uploading it here?

Alternatively, if your project is open source, pointing me to the project itself would be great :)

[danblae]

I unfortunately can not do that since it is neither private nor open source project and I am therefore not allowed to upload, what basically is, the entire documentation of the code. I will have a look myself if I can figure out which part of the XML it gets stuck on and maybe I can extract a snip of the xml it fails on and share that. This however might take a while since I won't be able to work of it for a week or so.

Thanks for the help and I will get back to it.

[marzer]

Ah, ok. That's unfortunate. Well, I'll tell you what the bug was in my case, which may help narrow it down:

An XML file containing the <compounddef kind="file"> for an actual code file entry (i.e. a @file directive) would now contain an <sectiondef kind="define">, which m.css wasn't expecting. My workaround was to just delete the <sectiondef>, since the #defines are also emitted elsewhere in the XML, so m.css worked fine with that. My suspicion is that there's some similar new XML nodes being emitted for other types.

[marzer]

Ah, ok, so it is definitely related to @file and @group. It seems that in 1.9.6 and below, doxygen would just emit identical XML for definitions at both the @file and @group level, but as of 1.9.7 they're having the @file entry act as a lightweight reference to the one in the @group, I guess as a means to reduce duplication. Makes sense I guess, but I wish they'd designed it that way to begin with! 😅

Here's a reduced example from my own codebase:

namespace muu
{
    /// @brief Allocates memory on a specific alignment boundary.
    /// @ingroup core
    void* aligned_alloc(size_t size, size_t alignment) noexcept;

    /// @brief Frees memory that was allocated using #muu::aligned_alloc().
    /// @ingroup core
    void aligned_free(void* ptr) noexcept;
}

XML generated by Doxygen 1.9.7:

<?xml version='1.0' encoding='UTF-8'?>
<doxygen xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="compound.xsd" version="1.9.7" xml:lang="en-US">
    <compounddef id="aligned__alloc_8h" kind="file" language="C++">
        <compoundname>aligned_alloc.h</compoundname>
        <innernamespace refid="namespacemuu">muu</innernamespace>
        <sectiondef kind="func">
            <member refid="group__core_1gad57726dc8177fff89f34e5049c060c8f" kind="function">
                <name>aligned_alloc</name>
            </member>
            <member refid="group__core_1ga980d079c5d73e2b8bf65a4130f3793b0" kind="function">
                <name>aligned_free</name>
            </member>
        </sectiondef>
        <briefdescription>
            <para>Contains the definition of <ref refid="group__core_1gad57726dc8177fff89f34e5049c060c8f" kindref="member">muu::aligned_alloc()</ref>. </para>
        </briefdescription>
        <detaileddescription></detaileddescription>
        <location file="muu/aligned_alloc.h"/>
    </compounddef>
</doxygen>

Compared to that generated by 1.9.6:

<?xml version='1.0' encoding='UTF-8'?>
<doxygen xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="compound.xsd" version="1.9.6" xml:lang="en-US">
    <compounddef id="aligned__alloc_8h" kind="file" language="C++">
        <compoundname>aligned_alloc.h</compoundname>
        <innernamespace refid="namespacemuu">muu</innernamespace>
        <sectiondef kind="func">
            <memberdef kind="function" id="group__core_1gad57726dc8177fff89f34e5049c060c8f" prot="public" static="no" const="no" explicit="no" inline="yes" noexcept="yes" virt="non-virtual">
                <type>void *</type>
                <definition>void * muu::aligned_alloc</definition>
                <argsstring>(size_t size, size_t alignment) noexcept</argsstring>
                <name>aligned_alloc</name>
                <qualifiedname>muu::aligned_alloc</qualifiedname>
                <param>
                    <type>size_t</type>
                    <declname>size</declname>
                </param>
                <param>
                    <type>size_t</type>
                    <declname>alignment</declname>
                </param>
                <briefdescription>
                    <para>Allocates memory on a specific alignment boundary. </para>
                </briefdescription>
                <detaileddescription></detaileddescription>
                <inbodydescription></inbodydescription>
                <location file="muu/aligned_alloc.h" line="33" column="13" bodyfile="muu/aligned_alloc.h" bodystart="33" bodyend="46"/>
            </memberdef>
            <memberdef kind="function" id="group__core_1ga980d079c5d73e2b8bf65a4130f3793b0" prot="public" static="no" const="no" explicit="no" inline="yes" noexcept="yes" virt="non-virtual">
                <type>void</type>
                <definition>void muu::aligned_free</definition>
                <argsstring>(void *ptr) noexcept</argsstring>
                <name>aligned_free</name>
                <qualifiedname>muu::aligned_free</qualifiedname>
                <param>
                    <type>void *</type>
                    <declname>ptr</declname>
                </param>
                <briefdescription>
                    <para>Frees memory that was allocated using <ref refid="group__core_1gad57726dc8177fff89f34e5049c060c8f" kindref="member">muu::aligned_alloc()</ref>. </para>
                </briefdescription>
                <detaileddescription>
                    <para>
                        <simplesect kind="warning">
                            <para>Do not use this to free memory that was not allocated using <ref refid="group__core_1gad57726dc8177fff89f34e5049c060c8f" kindref="member">muu::aligned_alloc()</ref>! </para>
                        </simplesect>
                    </para>
                </detaileddescription>
                <inbodydescription></inbodydescription>
                <location file="muu/aligned_alloc.h" line="53" column="6" bodyfile="muu/aligned_alloc.h" bodystart="53" bodyend="60"/>
            </memberdef>
        </sectiondef>
        <briefdescription>
            <para>Contains the definition of <ref refid="group__core_1gad57726dc8177fff89f34e5049c060c8f" kindref="member">muu::aligned_alloc()</ref>. </para>
        </briefdescription>
        <detaileddescription></detaileddescription>
        <location file="muu/aligned_alloc.h"/>
    </compounddef>
</doxygen>

Note that the 1.9.7 XML replaces the full <memberdef> with just <member>, which is the new bit that m.css doesn't understand.

I guess a full workaround in my caase would be to just not use @groups, but that's a bit shit 😅 I'm also not sure if this happens without using @group, it might just be that the @file one would defer to the <memberdef> in a namespace and still exhibit the same behaviour.

[eliaskosunen]

I also ran into this, while using poxy v0.15.0 + Doxygen v1.9.7, caused by Doxygen \groups. Minimal repro:

/**
 * My namespace.
 */
namespace ns {

/**
 * \defgroup my_group My group
 */

/**
 * My typedef.
 *
 * \ingroup my_group
 */
using my_typedef = void;

}

<!-- namespacens.xml -->
<?xml version='1.0' encoding='UTF-8'?>
<doxygen xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="compound.xsd" version="1.9.7" xml:lang="en-US">
    <compounddef id="namespacens" kind="namespace" language="C++">
        <compoundname>ns</compoundname>
        <sectiondef kind="typedef">
            <member refid="group__my__group_1ga42ef5f54d1178b862b27493f131aeb8b" kind="typedef">
                <name>my_typedef</name>
            </member>
        </sectiondef>
        <briefdescription></briefdescription>
        <detaileddescription>
            <para>My namespace. </para>
        </detaileddescription>
        <location file=".../header.h" line="4" column="1"/>
    </compounddef>
</doxygen>

Notably, the error vanishes after removing the \ingroup.

I guess the only real workaround, still, is downgrading Doxygen to v1.9.6?

[marzer]

@eliaskosunen Yeah I've just stuck with 1.9.6 for the time being. Haven't really had time to re-visit this. I could probably convert the 1.9.7 schema to the 1.9.6 one, but working with Doxygen's XML directly suuuuuuucks, so I've been putting it off 😅

[mosra]

Sorry for the nine month long delay. I don't know what to do here. The originating commit is https://github.com/doxygen/doxygen/commit/d460544167e38ce2e737ca4dd0c9dba37d5c19f3 which fixes a problem localized to some random downstream tool by making the XML output no longer self-contained within a file.

Which means, in order to keep the original functionality, where the file / namespace docs contain also all stuff for given group (like file docs contain all stuff for given namespace as well, etc), because THAT JUST MAKES SENSE TO HAVE, to be able to see what given header or namespace contains, I'd have to parse a XML while parsing a XML and that's just a big nope. Nononono.

The only reasonable solution I can think of is telling them to revert this crap and fix it differently, in the downstream tool. In the meantime I can of course work around this by ignoring the <member> bits, thus making the file docs empty and thus useless, but I'd really like to not start treating useless output as a standard here.

[mosra]

I'm going to tackle the workaround as part of #215, as it needs a regression test case which requires being able to test against at least some version of Doxygen newer than 1.9.7, and so far none of the versions were able to survive my tests well enough to be able to automate that on the CI.

[danblae]

I appreciate all the effort you put I to that. Unfortunately, I am not working with with either at the moment since I am not working on the project that needed it anymore.

I will keep an eye on the updates since I think I might use it again in the future.

Since I don't use it anymore; I don't see this as my issue anymore, so feel free to keep it open or close it at your discretion.

[mosra]

It's causing half of my tests to fail, so I have to fix that one way or another :)

[mosra]

A workaround (which essentially removes most of the content from file and namespace reference pages) is implemented in 09ddeebf4e77b8912bd8c914c5f75b25dd79d8a1. Once I calm down I'll try to convince them to revert this change. Until then, it's a choice between using grouping / modules or having actually useful file and namespace docs.

[marzer]

Yeah, if they'd at least have made it configurable, rather than just unconditionally and unilaterally changing their output schema, like that's not going to impact everyone consuming it...