visual studio debugger doesn't show STL data when using clang against C++20

[Inori]

Not sure if it's STL bug or Visual Studio bug: DevCom-10056375.

Apart from the bug, I want to know how this feature is controlled? Through some macros or autoexp.dat or something else?

[fsb4000]

Thank you, I confirm the issue.

Natvis error log:

Natvis: C:\Program Files\Microsoft Visual Studio\2022\Preview\Common7\Packages\Debugger\Visualizers\stl.natvis(1185,31): Successfully parsed expression '_Mypair._Myval2._Mysize' in type context 'std::list<int,std::allocator<int> >'.
Natvis: C:\Program Files\Microsoft Visual Studio\2022\Preview\Common7\Packages\Debugger\Visualizers\stl.natvis(1187,57): Successfully parsed expression '_Mypair' in type context 'std::list<int,std::allocator<int> >'.
Natvis: C:\Program Files\Microsoft Visual Studio\2022\Preview\Common7\Packages\Debugger\Visualizers\stl.natvis(1189,21): Successfully parsed expression '_Mypair._Myval2._Mysize' in type context 'std::list<int,std::allocator<int> >'.
Natvis: C:\Program Files\Microsoft Visual Studio\2022\Preview\Common7\Packages\Debugger\Visualizers\stl.natvis(1190,28): Error: class "std::_List_node<int,void *>" has no member "_Next"
    Error while evaluating '_Mypair._Myval2._Myhead->_Next' in the context of type 'TestCpp.exe!std::list<int,std::allocator<int>>'.
Natvis: C:\Program Files\Microsoft Visual Studio\2022\Preview\Common7\Packages\Debugger\Visualizers\stl.natvis(1184,4): Ignoring visualizer for type 'std::list<int,std::allocator<int> >' labeled as 'std::list<*>' because one or more sub-expressions was invalid.
Natvis: C:\Program Files\Microsoft Visual Studio\2022\Preview\Common7\Packages\Debugger\Visualizers\stl.natvis(1172,31): Error: identifier "_Mysize" is undefined
    Error while evaluating '_Mysize' in the context of type 'TestCpp.exe!std::list<int,std::allocator<int>>'.
Natvis: C:\Program Files\Microsoft Visual Studio\2022\Preview\Common7\Packages\Debugger\Visualizers\stl.natvis(1171,4): Ignoring visualizer for type 'std::list<int,std::allocator<int> >' labeled as 'std::list<*>' because one or more sub-expressions was invalid.

[Inori]

OK, just found that it is controlled by stl.natvis

[Inori]

Not only std::list, other containers also have this problem. I haven't tried all of them, but at least std::unordered_map and std::map have the same problem.

[fsb4000]

yes, I wanted to write the same. I tested std::set and found the same.

[Inori]

yes, I wanted to write the same. I tested std::set and found the same.

Is this easy to fix? If not, I'm going to fallback to C++17.

[fsb4000]

Is this easy to fix?

Maybe, but now I don't know what's wrong :( _List_node has _Next https://github.com/microsoft/STL/blob/60decd0d829e68666b556780002c83605d748d80/stl/inc/list#L286-L290

[Inori]

but now I don't know what's wrong :(

I'm not sure if it's related, but it seems some predefined macros are not correctly defined under clang.

See: https://developercommunity.visualstudio.com/t/Intellisense-not-work-correctly-on-clang/10054630

[fsb4000]

more recent clang-cl doesn't help :(

C:/tools/msys64/mingw64/bin/clang-cl.exe -v
clang version 14.0.3
Target: x86_64-pc-windows-msvc
Thread model: posix
InstalledDir: C:/tools/msys64/mingw64/bin

default(c++14) mode, debugger shows std::list correctly

C:/tools/msys64/mingw64/bin/clang-cl.exe main.cpp  /nologo /Z7 /link /natvis:STL.natvis
devenv /debugexe main.exe
// ctrl-o + choose main.cpp + setup break point

c++20, we have the same Natvis error :(

C:/tools/msys64/mingw64/bin/clang-cl.exe main.cpp /std:c++20 /nologo /Z7 /link /natvis:STL.natvis
devenv /debugexe main.exe
// ctrl-o + choose main.cpp + setup break point

[fsb4000]

clang-cl.exe main.cpp /nologo /Z7 /link
// renamed main.pdb to 14.pdb
clang-cl.exe main.cpp /std:c++20 /nologo /Z7 /link
// renamed main.pdb to 20.pdb

I downloaded https://github.com/Microsoft/microsoft-pdb/blob/master/cvdump/cvdump.exe

cvdump.exe 20.pdb > 20.txt
cvdump.exe 14.pdb > 14.txt

14.txt has two mentions class name = std::_List_node<int,void *>:

0x104a : Length = 78, Leaf = 0x1505 LF_STRUCTURE
    # members = 0,  field list type 0x0000, FORWARD REF, 
    Derivation list type 0x0000, VT shape type 0x0000
    Size = 0, class name = std::_List_node<int,void *>, unique name = .?AU?$_List_node@HPEAX@std@@, UDT(0x000010db) 

0x10db : Length = 78, Leaf = 0x1505 LF_STRUCTURE
    # members = 7,  field list type 0x10da, CONSTRUCTOR, CONTAINS NESTED, 
    Derivation list type 0x0000, VT shape type 0x0000
    Size = 24, class name = std::_List_node<int,void *>, unique name = .?AU?$_List_node@HPEAX@std@@, UDT(0x000010db)

20.txt has only one:

0x104a : Length = 78, Leaf = 0x1505 LF_STRUCTURE
    # members = 0,  field list type 0x0000, FORWARD REF, 
    Derivation list type 0x0000, VT shape type 0x0000
    Size = 0, class name = std::_List_node<int,void *>, unique name = .?AU?$_List_node@HPEAX@std@@

[fsb4000]

I reduced:

template <class Value_type>
struct node { // list node
    node* _Next; // successor node, or first element if head
    node* _Prev; // predecessor node, or last element if head

    node(const node&) = delete;
    node& operator=(const node&) = delete;

    static node* Buyheadnode(void* buf) {
        node* _Result = (node*)buf;
        return _Result;
    }
};

int main()
{
    alignas(alignof(void*)) char buf[128]{};
    node<int>* l = node<int>::Buyheadnode(buf);
    return 0;
}

clang-cl with C++14 or C++17, the debugger is OK:

clang-cl with C++20 is not OK

Also if we remove

    node(const node&) = delete;
    node& operator=(const node&) = delete;

then the debugger is OK in C++20

[fsb4000]

I created LLVM-55768

[StephanTLavavej]

1. What happens if we give the node type a defaulted default constructor?

2. It looks like the issue might be that we never actually construct the node, just some of its data members. It might be possible to invasively change this, by putting the value into a union, although that's near the maximum ABI risk we'd want to take. (If we actually construct and destroy the node type, we need some way to avoid constructing/destroying the value for the sentinel node.)

[frederick-vs-ja]

1. What happens if we give the node type a defaulted default constructor? 2. It looks like the issue might be that we never actually construct the node, just some of its data members. It might be possible to invasively change this, by putting the value into a union, although that's near the maximum ABI risk we'd want to take. (If we actually construct and destroy the node type, we need some way to avoid constructing/destroying the value for the sentinel node.)

The reason is seemly concluded that the node type failed to be an aggregate (implicit-lifetime) type and then the lifetime analysis is being problematic. Note that _List_node has an explicitly deleted copy constructor, which makes it an aggregate in C++14/17, but not in C++20 and later.

The resolution might be explicitly deleting the move assignment operator instead of the copy constructor (it'll be implicitly deleted, and the move constructor will be suppressed). The default constructor will be implicitly declared as a side effect, but IMO it's fine.

[fsb4000]

@Inori

A workaround is found: add -fstandalone-debug for clang-cl

[fsb4000]

@StephanTLavavej yes, it seems default constructor helps. I tested only reduced example not rebuild STL and look what dwblaikie suggested: https://github.com/llvm/llvm-project/issues/55768#issuecomment-1141537838

[Inori]

Yes, I saw it in LLVM issue tracker. Thanks for all of you guys.

Not sure if I should close this issue, you can close this at any time you need. @fsb4000

[StephanTLavavej]

I think we'd accept a PR to add a defaulted default constructor to this node type, with a comment citing this issue, as long as the end-to-end scenario with clang-cl has been verified. Additionally, we should check whether the other sentinel node types are affected (forward_list and the map family of containers come to mind).

The more "correct" but invasive change of modifying the node to use a union would pose problems for C++/CX (we have workarounds elsewhere in the codebase); that seems like far too much risk/effort for a minor debugging issue.

[frederick-vs-ja]

All node containers (forward_list, list, map, unordered_map, etc.) are affected.

Unfortunately, I've found that currently clang (sometimes?) doesn't emit debug info if the node is not actually constructed. We can put each data member into an anonymous union and add constructor/destructor that are no-op...

[StephanTLavavej]

It might be possible to achieve this without introducing a union - since we can construct and destroy real nodes, and we only have to do the weird partial construction/destruction for the sentinel node. As long as we construct any node for real, that should emit the proper debug info.

[frederick-vs-ja]

BTW, I attempted to make MSVC STL's std::(multi)map constexpr, and found that everything can work except for node handle's key() (which requires CWG-2514 to be resolved).

According to the experience of constexpr-ization, I guess at least for associative containers, the proper fix should be adding a constructor that constructs the node, except that the element object is wrapped in a union and not constructed by the constructor.