[preprocessor] Clang doesn't warn about preprocessing directives in function-like macro arguments

[StephanTLavavej]

N4964 [cpp.replace.general]/13:

The sequence of preprocessing tokens bounded by the outside-most matching parentheses forms the list of arguments for the function-like macro. The individual arguments within the list are separated by comma preprocessing tokens, but comma preprocessing tokens between matching inner parentheses do not separate arguments. If there are sequences of preprocessing tokens within the list of arguments that would otherwise act as preprocessing directives, the behavior is undefined.

When preprocessing directives are embedded in function-like macro arguments, MSVC appears to reject with its traditional (horribly deficient) preprocessor, and emits a dedicated warning with its modern (more conformant) preprocessor (and then later rejects, at least some of the time). Clang appears to silently accept, which is surprising.

C:\Temp>type meow.cpp

#include <cstdio>

#define OUTER_MACRO(ARG) std::puts(ARG)

int main() {
    OUTER_MACRO("I have "
#ifdef MANY
                "1729"
#else
                 "5"
#endif
                " cute fluffy kittens.");
}

C:\Temp>cl /EHsc /nologo /W4 meow.cpp && meow
meow.cpp
meow.cpp(12): error C2121: '#': invalid character: possibly the result of a macro expansion
meow.cpp(6): error C2146: syntax error: missing ')' before identifier 'ifdef'
meow.cpp(6): error C2146: syntax error: missing ';' before identifier 'ifdef'
meow.cpp(6): error C2065: 'ifdef': undeclared identifier
meow.cpp(6): error C2146: syntax error: missing ';' before identifier 'MANY'
meow.cpp(6): error C2065: 'MANY': undeclared identifier
meow.cpp(6): error C2143: syntax error: missing ';' before 'string'
meow.cpp(6): error C2143: syntax error: missing ';' before 'else'
meow.cpp(12): error C2181: illegal else without matching if
meow.cpp(6): error C2146: syntax error: missing ';' before identifier 'endif'
meow.cpp(6): error C2065: 'endif': undeclared identifier
meow.cpp(6): error C2059: syntax error: ')'

C:\Temp>cl /EHsc /nologo /W4 /Zc:preprocessor meow.cpp && meow
meow.cpp
meow.cpp(7): warning C5101: use of preprocessor directive in function-like macro argument list is undefined behavior
meow.cpp(9): warning C5101: use of preprocessor directive in function-like macro argument list is undefined behavior
meow.cpp(11): warning C5101: use of preprocessor directive in function-like macro argument list is undefined behavior
meow.cpp(6): error C2143: syntax error: missing ')' before '#'
meow.cpp(6): error C2143: syntax error: missing ';' before '#'
meow.cpp(6): error C2059: syntax error: '#'
meow.cpp(6): error C2059: syntax error: ')'

C:\Temp>clang-cl /EHsc /nologo /W4 meow.cpp && meow
I have 5 cute fluffy kittens.

C:\Temp>clang-cl -v
clang version 16.0.5
Target: x86_64-pc-windows-msvc
Thread model: posix
InstalledDir: C:\Program Files\Microsoft Visual Studio\2022\Preview\VC\Tools\Llvm\x64\bin

This allowed UB to accumulate in libc++'s test suite, see #73440.

[shafik]

CC @AaronBallman @cor3ntin

[llvmbot]

@llvm/issue-subscribers-clang-frontend

Author: Stephan T. Lavavej (StephanTLavavej)

[N4964](https://isocpp.org/files/papers/N4964.pdf) \[cpp.replace.general\]/13: > The sequence of preprocessing tokens bounded by the outside-most matching parentheses forms the list of arguments for the function-like macro. The individual arguments within the list are separated by comma preprocessing tokens, but comma preprocessing tokens between matching inner parentheses do not separate arguments. If there are sequences of preprocessing tokens within the list of arguments that would otherwise act as preprocessing directives, the behavior is undefined. When preprocessing directives are embedded in function-like macro arguments, MSVC appears to reject with its traditional (horribly deficient) preprocessor, and emits a dedicated warning with its modern (more conformant) preprocessor (and then later rejects, at least some of the time). Clang appears to silently accept, which is surprising. ``` C:\Temp>type meow.cpp ``` ```cpp #include <cstdio> #define OUTER_MACRO(ARG) std::puts(ARG) int main() { OUTER_MACRO("I have " #ifdef MANY "1729" #else "5" #endif " cute fluffy kittens."); } ``` ``` C:\Temp>cl /EHsc /nologo /W4 meow.cpp && meow meow.cpp meow.cpp(12): error C2121: '#': invalid character: possibly the result of a macro expansion meow.cpp(6): error C2146: syntax error: missing ')' before identifier 'ifdef' meow.cpp(6): error C2146: syntax error: missing ';' before identifier 'ifdef' meow.cpp(6): error C2065: 'ifdef': undeclared identifier meow.cpp(6): error C2146: syntax error: missing ';' before identifier 'MANY' meow.cpp(6): error C2065: 'MANY': undeclared identifier meow.cpp(6): error C2143: syntax error: missing ';' before 'string' meow.cpp(6): error C2143: syntax error: missing ';' before 'else' meow.cpp(12): error C2181: illegal else without matching if meow.cpp(6): error C2146: syntax error: missing ';' before identifier 'endif' meow.cpp(6): error C2065: 'endif': undeclared identifier meow.cpp(6): error C2059: syntax error: ')' C:\Temp>cl /EHsc /nologo /W4 /Zc:preprocessor meow.cpp && meow meow.cpp meow.cpp(7): warning C5101: use of preprocessor directive in function-like macro argument list is undefined behavior meow.cpp(9): warning C5101: use of preprocessor directive in function-like macro argument list is undefined behavior meow.cpp(11): warning C5101: use of preprocessor directive in function-like macro argument list is undefined behavior meow.cpp(6): error C2143: syntax error: missing ')' before '#' meow.cpp(6): error C2143: syntax error: missing ';' before '#' meow.cpp(6): error C2059: syntax error: '#' meow.cpp(6): error C2059: syntax error: ')' C:\Temp>clang-cl /EHsc /nologo /W4 meow.cpp && meow I have 5 cute fluffy kittens. C:\Temp>clang-cl -v clang version 16.0.5 Target: x86_64-pc-windows-msvc Thread model: posix InstalledDir: C:\Program Files\Microsoft Visual Studio\2022\Preview\VC\Tools\Llvm\x64\bin ``` This allowed UB to accumulate in libc++'s test suite, see #73440.

[cor3ntin]

This recently came up in WG21 as part of P2843 and https://github.com/cplusplus/papers/issues/1413

The undefined behavior in the spec is how historically C allowed implementation divergence.

I think we should wait for WG21 to decide whether specific directives should be allowed/removed. If my recall is correct, clang is consistent with GCC. Adding a warning now might lead to too much churn and noise.

I struggle to find the minutes of that specific discussion though.

@erichkeane

[erichkeane]

@cor3ntin : all of the discussion on that paper happened in "SG12", so we put it on that wiki: https://wiki.edg.com/bin/view/Wg21kona2023/SG12