clang-cl preprocessor error where VC++ works

[Quuxplusone]

Bugzilla Link	PR27380
Status	NEW
Importance	P normal
Reported by	Edward Diener (eldlistmailingz@tropicsoft.com)
Reported on	2016-04-15 16:48:10 -0700
Last modified on	2016-04-16 00:06:57 -0700
Version	trunk
Hardware	PC Windows NT
CC	dgregor@apple.com, llvm-bugs@lists.llvm.org, marci_r@web.de, s_bugzilla@nedprod.com
Fixed by commit(s)
Attachments
Blocks
Blocked by
See also	PR27169

The following code compiles without error using VC++ 14 but gives errors using
the latest clang-cl from the 'trunk':

#define TEST_MACRO(x,y) TEST_EXPAND(2, 0, TEST_SOMETHING(x,y,1))
#define TEST_SOMETHING(x,y,z) 2

#define TEST_EXPAND(...) TEST_EXPAND_I(TEST_OVR(TEST_NAME, __VA_ARGS__),
(__VA_ARGS__))
#define TEST_OVR(name,...) TEST_SOMENAME
#define TEST_EXPAND_I(m, args) TEST_EXPAND_II(m, args)
#define TEST_EXPAND_II(m, args) TEST_CAT(m ## args,)
#define TEST_SOMENAME(x,y,z) ;

#define TEST_CAT(a, b) TEST_CAT_I(a, b)
#define TEST_CAT_I(a, b) TEST_CAT_II(~, a ## b)
#define TEST_CAT_II(p, res) res

int main()
    {
    TEST_MACRO(1,2)
    }

---------------------------------------------------------------------------

The clang errors are:

test_clang.cpp(17,2):  error: pasting formed 'TEST_SOMENAME(', an invalid
preprocessing token [-Winvalid-token-paste]
        TEST_MACRO(1,2)
        ^
test_clang.cpp(2,25):  note: expanded from macro 'TEST_MACRO'
#define TEST_MACRO(x,y) TEST_EXPAND(2, 0, TEST_SOMETHING(x,y,1))
                        ^
test_clang.cpp(5,26):  note: expanded from macro 'TEST_EXPAND'
#define TEST_EXPAND(...) TEST_EXPAND_I(TEST_OVR(TEST_NAME, __VA_ARGS__),
(__VA_ARGS__))
                         ^
test_clang.cpp(7,32):  note: expanded from macro 'TEST_EXPAND_I'
#define TEST_EXPAND_I(m, args) TEST_EXPAND_II(m, args)
                               ^
test_clang.cpp(8,44):  note: expanded from macro 'TEST_EXPAND_II'
#define TEST_EXPAND_II(m, args) TEST_CAT(m ## args,)

----------------------------------------------------------------------------

The clang-cl options are:

-TP /Od /Ob0 /W3 /GR /MDd  /Zc:forScope /Zc:wchar_t -fmsc-version=1900 /wd4675
/EHs -fmacro-backtrace-limit=0 -c

-----------------------------------------------------------------------------

The VC++14 options when compiling successfully are:

/Zm800 -TP /Z7 /Od /Ob0 /W4 /GR /MDd /Zc:forScope /Zc:wchar_t /wd4675 /EHs -c

---------------------------------------------------------------------------

This example is a simplified version of a failure which occurs when running the
Boost Preprocessor library tests with clang-cl, where the Boost Preprocessor
library is setup to use the exact same workarounds for clang-cl as it does for
VC++, since neither is a C++ standard conforming preprocessor.

A possible fix for this problem is not a change in the code above, even if some
specific change in the code will get clang-cl to work. If clang-cl is truly
emulating the non-standard VC++ preprocessor it must be able to compile
preprocessor code which VC++ also compiles without errors. Thus I am reporting
this problem here.

[Quuxplusone]

Related: An alternative to making the MSVC preprocessor emulation more like MSVC is to give MSVC ABI target users the ability to choose an ISO standard preprocessor instead whilst retaining the MSVC ABI target: https://llvm.org/bugs/show_bug.cgi?id=27169

[Quuxplusone]

If the '-Wno-invalid-token-paste' option is passed to clang-cl when compiling, the code compiles without errors. The '-Wno-invalid-token-paste' should be the default when targeting VC++ since the VC++ preprocessor works as if '-Wno-invalid-token-paste' were in effect and clang targeting VC++ is emulating the VC++ preprocessor with its non-standard behavior.