llvmbot
commented
2 months ago @llvm/issue-subscribers-clang-frontend
Author: Scott McPeak (smcpeak)
`ClassTemplatePartialSpecializationDecl` inherits `ClassTemplateSpecializationDecl`, which, for an explicit specialization, contains the `TypeAsWritten` pointer to a `TypeSourceInfo` (inside `ExplicitSpecializationInfo`).
When `RecursiveASTVisitor` traverses a non-partial `ClassTemplateSpecializationDecl`, it traverses `TypeAsWritten`. Excerpt of `RecursiveASTVisitor.h`:
```C++
#define DEF_TRAVERSE_TMPL_SPEC_DECL(TMPLDECLKIND, DECLKIND) \
DEF_TRAVERSE_DECL(TMPLDECLKIND##TemplateSpecializationDecl, { \
/* For implicit instantiations ("set<int> x;"), we don't want to \
recurse at all, since the instatiated template isn't written in \
the source code anywhere. (Note the instatiated *type* -- \
set<int> -- is written, and will still get a callback of \
TemplateSpecializationType). For explicit instantiations \
("template set<int>;"), we do need a callback, since this \
is the only callback that's made for this instantiation. \
We use getTypeAsWritten() to distinguish. */ \
if (TypeSourceInfo *TSI = D->getTypeAsWritten()) \
TRY_TO(TraverseTypeLoc(TSI->getTypeLoc())); \
[...]
```
However, when it traverses a `ClassTemplatePartialSpecializationDecl`, the `TypeAsWritten` is not traversed. Instead, it traverses the template arguments that are logically part of the `TypeAsWritten`, but not `TypeAsWritten` itself:
```C++
#define DEF_TRAVERSE_TMPL_PART_SPEC_DECL(TMPLDECLKIND, DECLKIND) \
DEF_TRAVERSE_DECL(TMPLDECLKIND##TemplatePartialSpecializationDecl, { \
/* The partial specialization. */ \
if (TemplateParameterList *TPL = D->getTemplateParameters()) { \
for (TemplateParameterList::iterator I = TPL->begin(), E = TPL->end(); \
I != E; ++I) { \
TRY_TO(TraverseDecl(*I)); \
} \
} \
/* The args that remains unspecialized. */ \
TRY_TO(TraverseTemplateArgumentLocsHelper( \
D->getTemplateArgsAsWritten()->getTemplateArgs(), \
D->getTemplateArgsAsWritten()->NumTemplateArgs)); \
\
/* Don't need the *TemplatePartialSpecializationHelper, even \
though that's our parent class -- we already visit all the \
template args here. */ \
TRY_TO(Traverse##DECLKIND##Helper(D)); \
\
/* Instantiations will have been visited with the primary template. */ \
})
```
In `ClassTemplatePartialSpecializationDecl`, `ArgsAsWritten` is a separate field from `TypeAsWritten`, even though the former is logically a part of the latter. There is also a relevant `FIXME` comment:
```C++
class ClassTemplatePartialSpecializationDecl
: public ClassTemplateSpecializationDecl {
/// The list of template parameters
TemplateParameterList* TemplateParams = nullptr;
/// The source info for the template arguments as written.
/// FIXME: redundant with TypeAsWritten?
const ASTTemplateArgumentListInfo *ArgsAsWritten = nullptr;
```
Since `TypeAsWritten` is part of `ClassTemplatePartialSpecializationDecl`, and is traversed for a non-partial `ClassTemplateSpecializationDecl`, I think `RecursiveASTVisitor` should also traverse it for partial specializations.
Fixing this could be tricky though, since merely traversing `TypeAsWritten` in addition to `ArgsAsWritten` would entail traversing the arguments twice. Ideally, `ArgsAsWritten` would be removed and `TypeAsWritten` used instead to get the arguments, although I do not know what other consequences that might have. Less ambitiously, `RecursiveASTVisitor` could be changed to traverse `TypeAsWritten` instead of `ArgsAsWritten`.
Below is a complete program demonstrating the problem by printing the `Decl` and `TypeLoc` nodes traversed by `RecursiveASTVisitor`, along with the `TypeAsWritten` for class template specializations for comparison:
```C++
// rav-misses-ctpsd-typeloc.cc
// Demonstrate RAV failing to visit the TypeLoc in
// ClassTemplatePartialSpecializationDecl.
// clang
#include "clang/AST/RecursiveASTVisitor.h" // clang::RecursiveASTVisitor
#include "clang/Frontend/ASTUnit.h" // clang::ASTUnit
#include "clang/Frontend/CompilerInstance.h" // clang::CompilerInstance
#include "clang/Frontend/CompilerInvocation.h" // clang::CompilerInvocation
#include "clang/Frontend/Utils.h" // clang::createInvocation
#include "clang/Serialization/PCHContainerOperations.h" // clang::PCHContainerOperations
// libc++
#include <iostream> // std::ostream
#include <sstream> // std::ostringstream
#include <string> // std::string
using clang::dyn_cast;
// Use RAV to print AST nodes.
class RAVPrinterVisitor : public clang::RecursiveASTVisitor<RAVPrinterVisitor> {
public: // types
// Name of the base class visitor, to make it easier to call its
// methods from the same-named methods when overridden.
typedef clang::RecursiveASTVisitor<RAVPrinterVisitor> BaseClass;
public: // data
// AST context.
clang::ASTContext &m_astContext;
// Number of levels of indentation to print.
int m_indentLevel;
// Stream to print to.
std::ostream &m_os;
public: // methods
RAVPrinterVisitor(std::ostream &os, clang::ASTContext &astContext)
: BaseClass(),
m_astContext(astContext),
m_indentLevel(0),
m_os(os)
{}
// Indentation string corresponding to 'm_indentLevel'.
std::string indentString() const;
// Render 'loc' as a string.
std::string locStr(clang::SourceLocation loc) const;
// RecursiveASTVisitor customization. Get everything RAV can.
bool shouldVisitTemplateInstantiations() const { return true; }
bool shouldVisitImplicitCode() const { return true; }
// RecursiveASTVisitor methods.
bool TraverseDecl(clang::Decl *decl);
bool TraverseTypeLoc(clang::TypeLoc TL);
};
std::string RAVPrinterVisitor::indentString() const
{
std::ostringstream oss;
for (int i=0; i < m_indentLevel; ++i) {
oss << " ";
}
return oss.str();
}
std::string RAVPrinterVisitor::locStr(clang::SourceLocation loc) const
{
return loc.printToString(m_astContext.getSourceManager());
}
bool RAVPrinterVisitor::TraverseDecl(clang::Decl *decl)
{
if (!decl) {
return true;
}
m_os << indentString()
<< decl->getDeclKindName() << "Decl at "
<< locStr(decl->getBeginLoc()) << "\n";
++m_indentLevel;
bool ret = BaseClass::TraverseDecl(decl);
// For both full and partial specializations, print the type as
// written, for comparison with what was found via RAV above.
if (auto ctsd = dyn_cast<clang::ClassTemplateSpecializationDecl>(decl)) {
m_os << "---- BEGIN expected TypeAsWritten ----\n";
if (auto tsi = ctsd->getTypeAsWritten()) {
TraverseTypeLoc(tsi->getTypeLoc());
}
else {
m_os << "(none)\n";
}
m_os << "---- END expected TypeAsWritten ----\n";
}
--m_indentLevel;
return ret;
}
bool RAVPrinterVisitor::TraverseTypeLoc(clang::TypeLoc typeLoc)
{
m_os << indentString()
<< "TypeLoc(" << typeLoc.getTypePtr()->getTypeClassName() << ")"
<< " at " << locStr(typeLoc.getBeginLoc()) << "\n";
++m_indentLevel;
bool ret = BaseClass::TraverseTypeLoc(typeLoc);
--m_indentLevel;
return ret;
}
void ravPrinterVisitorTU(std::ostream &os,
clang::ASTContext &astContext)
{
RAVPrinterVisitor rpv(os, astContext);
rpv.TraverseAST(astContext);
}
// This is boilerplate up to running the visitor.
int main(int argc, char **argv)
{
std::vector<char const *> commandLine;
commandLine.push_back(CLANG_LLVM_INSTALL_DIR "/bin/clang");
for (int i = 1; i < argc; ++i) {
commandLine.push_back(argv[i]);
}
std::shared_ptr<clang::CompilerInvocation> compilerInvocation(
clang::createInvocation(llvm::ArrayRef(commandLine)));
if (!compilerInvocation) {
return 2;
}
std::shared_ptr<clang::PCHContainerOperations> pchContainerOps(
new clang::PCHContainerOperations());
clang::DiagnosticOptions *diagnosticOptions =
&(compilerInvocation->getDiagnosticOpts());
clang::IntrusiveRefCntPtr<clang::DiagnosticsEngine> diagnosticsEngine(
clang::CompilerInstance::createDiagnostics(
diagnosticOptions /*callee takes refcount ownership*/));
std::unique_ptr<clang::ASTUnit> ast(
clang::ASTUnit::LoadFromCompilerInvocationAction(
compilerInvocation,
pchContainerOps,
diagnosticsEngine));
if (ast == nullptr || diagnosticsEngine->getNumErrors() > 0) {
return 2;
}
ravPrinterVisitorTU(std::cout, ast->getASTContext());
return 0;
}
// EOF
```
`Makefile`:
```make
# rav-misses-ctpsd-typeloc/Makefile
# --------------------------- Configuration ----------------------------
# Set to 1 if I am using a build from source, 0 for a binary
# distribution.
USE_SOURCE_BUILD := 1
ifeq ($(USE_SOURCE_BUILD),1)
# Trying my own build.
CLANG_LLVM_SRC_DIR = $(HOME)/bld/llvm-project-18.1.4
CLANG_LLVM_INSTALL_DIR = $(CLANG_LLVM_SRC_DIR)/build
else
# Installation directory from a binary distribution.
# Has five subdirectories: bin include lib libexec share.
CLANG_LLVM_INSTALL_DIR = $(HOME)/opt/clang+llvm-16.0.0-x86_64-linux-gnu-ubuntu-18.04
endif
# Let the user override my defaults.
-include pre-config.mk
# --------------------- llvm-config query results ----------------------
# Program to query the various LLVM configuration options.
LLVM_CONFIG := $(CLANG_LLVM_INSTALL_DIR)/bin/llvm-config
# C++ compiler options to ensure ABI compatibility.
LLVM_CXXFLAGS := $(shell $(LLVM_CONFIG) --cxxflags)
# Directory containing the clang library files, both static and dynamic.
LLVM_LIBDIR := $(shell $(LLVM_CONFIG) --libdir)
# Other flags needed for linking, whether statically or dynamically.
LLVM_LDFLAGS_AND_SYSTEM_LIBS := $(shell $(LLVM_CONFIG) --ldflags --system-libs)
# -------------------------- Compiler options --------------------------
# C++ compiler.
CXX = $(CLANG_LLVM_INSTALL_DIR)/bin/clang++
# Compiler options, including preprocessor options.
CXXFLAGS =
CXXFLAGS += -Wall
CXXFLAGS += -Werror
CXXFLAGS += $(LLVM_CXXFLAGS)
ifeq ($(USE_SOURCE_BUILD),1)
# When using my own build, I need to separately point at clang includes.
CXXFLAGS += -I$(CLANG_LLVM_SRC_DIR)/clang/include
CXXFLAGS += -I$(CLANG_LLVM_INSTALL_DIR)/tools/clang/include
endif
# Tell the source code where the clang installation directory is.
CXXFLAGS += -DCLANG_LLVM_INSTALL_DIR='"$(CLANG_LLVM_INSTALL_DIR)"'
# Switch to enable creation of .d files.
GENDEPS_FLAGS = -MMD
# Linker options.
LDFLAGS =
# Pull in clang+llvm via libclang-cpp.so, which has everything, but is
# only available as a dynamic library.
LDFLAGS += -lclang-cpp
# Arrange for the compiled binary to search the libdir for that library.
# Otherwise, one can set the LD_LIBRARY_PATH envvar before running it.
# Note: the -rpath switch does not work on Windows.
LDFLAGS += -Wl,-rpath=$(LLVM_LIBDIR)
# It appears that llvm::raw_os_ostream::~raw_os_ostream is missing from
# libclang-cpp, so I have to link with LLVMSupport statically.
LDFLAGS += -lLLVMSupport
# Get the needed -L search path, plus things like -ldl.
LDFLAGS += $(LLVM_LDFLAGS_AND_SYSTEM_LIBS)
# Optional custom modifications.
-include config.mk
# ------------------------------ Recipes -------------------------------
# Default target.
all:
.PHONY: all
# Pull in automatic dependencies.
-include $(wildcard *.d)
# Compile a C++ source file.
%.o: %.cc
$(CXX) -c -o $@ $(GENDEPS_FLAGS) $(CXXFLAGS) $<
OBJS :=
OBJS += rav-misses-ctpsd-typeloc.o
# Executable.
all: rav-misses-ctpsd-typeloc.exe
rav-misses-ctpsd-typeloc.exe: $(OBJS)
$(CXX) -g -Wall -o $@ $(OBJS) $(LDFLAGS)
# ---------------------------- 'run' target ----------------------------
.PHONY: run
run: rav-misses-ctpsd-typeloc.exe
./rav-misses-ctpsd-typeloc.exe test.cc
# --------------------------- 'clean' target ---------------------------
.PHONY: clean
clean:
$(RM) *.o *.d *.exe
$(RM) -r out
# EOF
```
Test input:
```C++
// test.cc
// Input for rav-misses-ctpsd-typeloc.exe.
template <typename T>
struct S;
// ClassTemplatePartialSpecializationDecl, for which the TypeAsWritten
// is not printed by RAV.
template <typename U>
struct S<U*>;
// ^^^^^ TypeAsWritten
// ClassTemplateSpecializationDecl, for which the TypeAsWritten *is*
// printed by RAV.
template <>
struct S<int>;
// ^^^^^^ TypeAsWritten
// EOF
```
Annotated example run:
```plain-text
$ make run
./rav-misses-ctpsd-typeloc.exe test.cc
TranslationUnitDecl at <invalid loc>
TypedefDecl at <invalid loc>
TypeLoc(Builtin) at <invalid loc>
TypedefDecl at <invalid loc>
TypeLoc(Builtin) at <invalid loc>
TypedefDecl at <invalid loc>
TypeLoc(Record) at <invalid loc>
TypedefDecl at <invalid loc>
TypeLoc(Pointer) at <invalid loc>
TypeLoc(Builtin) at <invalid loc>
TypedefDecl at <invalid loc>
TypeLoc(ConstantArray) at <invalid loc>
TypeLoc(Record) at <invalid loc>
ClassTemplateDecl at test.cc:4:1
TemplateTypeParmDecl at test.cc:4:11
CXXRecordDecl at test.cc:5:1
ClassTemplatePartialSpecializationDecl at test.cc:9:1
TemplateTypeParmDecl at test.cc:9:11
TypeLoc(Pointer) at test.cc:10:10 # actual (wrong)
TypeLoc(TemplateTypeParm) at test.cc:10:10 # actual (wrong)
---- BEGIN expected TypeAsWritten ----
TypeLoc(TemplateSpecialization) at test.cc:10:8 # expected
TypeLoc(Pointer) at test.cc:10:10 # expected
TypeLoc(TemplateTypeParm) at test.cc:10:10 # expected
---- END expected TypeAsWritten ----
ClassTemplateSpecializationDecl at test.cc:15:1
TypeLoc(TemplateSpecialization) at test.cc:16:8 # actual (right)
TypeLoc(Builtin) at test.cc:16:10 # actual (right)
---- BEGIN expected TypeAsWritten ----
TypeLoc(TemplateSpecialization) at test.cc:16:8 # expected
TypeLoc(Builtin) at test.cc:16:10 # expected
---- END expected TypeAsWritten ----
```
I see the above behavior with clang-16.0.0 from a binary distribution and clang-18.1.4 built myself.
(I'm just reporting this, I don't intend to try to fix it myself.)
ClassTemplatePartialSpecializationDeclinheritsClassTemplateSpecializationDecl, which, for an explicit specialization, contains theTypeAsWrittenpointer to aTypeSourceInfo(insideExplicitSpecializationInfo).When
RecursiveASTVisitortraverses a non-partialClassTemplateSpecializationDecl, it traversesTypeAsWritten. Excerpt ofRecursiveASTVisitor.h:However, when it traverses a
ClassTemplatePartialSpecializationDecl, theTypeAsWrittenis not traversed. Instead, it traverses the template arguments that are logically part of theTypeAsWritten, but notTypeAsWrittenitself:In
ClassTemplatePartialSpecializationDecl,ArgsAsWrittenis a separate field fromTypeAsWritten, even though the former is logically a part of the latter. There is also a relevantFIXMEcomment:Since
TypeAsWrittenis part ofClassTemplatePartialSpecializationDecl, and is traversed for a non-partialClassTemplateSpecializationDecl, I thinkRecursiveASTVisitorshould also traverse it for partial specializations.Fixing this could be tricky though, since merely traversing
TypeAsWrittenin addition toArgsAsWrittenwould entail traversing the arguments twice. Ideally,ArgsAsWrittenwould be removed andTypeAsWrittenused instead to get the arguments, although I do not know what other consequences that might have. Less ambitiously,RecursiveASTVisitorcould be changed to traverseTypeAsWritteninstead ofArgsAsWritten.Below is a complete program demonstrating the problem by printing the
DeclandTypeLocnodes traversed byRecursiveASTVisitor, along with theTypeAsWrittenfor class template specializations for comparison:Makefile:Test input:
Annotated example run:
I see the above behavior with clang-16.0.0 from a binary distribution and clang-18.1.4 built myself.
(I'm just reporting this, I don't intend to try to fix it myself.)