Preprocessor failure evaluating BOOST_WORKAROUND

[guwirth]

Community C++ plugin 0.9.1, getting error message:

17:26:46.716 WARN  - Error evaluating expression '( ( 1600 _WORKAROUND_GUARD + 0 == 0 ) && ( 1600 != 0 ) && ( 1 % ( ( ( 1600 < 1300 ) ) + 1 ) ) )', assuming 0

Source: ...\Boost\1.49.0\include\boost\detail\workaround.hpp, line 233

#  define BOOST_WORKAROUND(symbol, test)                \
         ((symbol ## _WORKAROUND_GUARD + 0 == 0) &&     \
         (symbol != 0) && (1 % (( (symbol test) ) + 1)))

[wenns]

I see, youre testing with Boost. Thats a hell of a C++ project, isnt it ;)

I did a while ago too, and I remember this parse failure. Its not a regression. Also, when I remember correctly, its a quite complicated failure, which doesnt happen often.

[guwirth]

Some other boost 1.49 problems below. Keep in mind that 1.49 is still using C++03! Beside this problems other code could be parsed without syntax errors (at least in my example).

17:32:08.617 WARN  - [C:\Program Files (x86)\Boost\1.49.0\include\boost\preprocessor\iteration\detail\iter\forward1.hpp:47]: cannot parse included filename: BOOST_PP_TUPLE_ELEM_2(0,10,<boost/utility/detail/result_of_iterate.hpp>,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)'
17:32:08.617 WARN  - [C:\Program Files (x86)\Boost\1.49.0\include\boost\preprocessor\iteration\detail\iter\forward1.hpp:47]: cannot find the sources for '#        include BOOST_PP_FILENAME_1'
17:26:50.198 WARN  - Error evaluating expression '( ( 0 _WORKAROUND_GUARD + 0 == 0 ) && ( 0 != 0 ) && ( 1 % ( ( ( 0 != ( ( 0x0304 ) - ( 0x0304 ) ) ) ) + 1 ) ) )', assuming 0
17:26:21.272 WARN  - [C:\Program Files (x86)\Boost\1.49.0\include\boost\mpl\aux_\include_preprocessed.hpp:37]: cannot find the sources for '#   include BOOST_PP_STRINGIZE(boost/mpl/aux_/preprocessed/AUX778076_PREPROCESSED_HEADER)'
17:26:23.661 WARN  - Cannot decode the number '0xffffffffffffffffu' falling back to max long (9223372036854775807) instead

[wenns]

On 06/05/2014 03:52 PM, Günter Wirth wrote:

Some other boost 1.49 problems below. Keep in mind that 1.49 is still using C++03! Beside this problems other code could be parsed without syntax errors (at least in my example).

|17:32:08.617 WARN - [C:\Program Files (x86)\Boost\1.49.0\include\boost\preprocessor\iteration\detail\iter\forward1.hpp:47]: cannot parse included filename: BOOST_PP_TUPLE_ELEM_2(0,10,<boost/utility/detail/result_of_iterate.hpp>,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)'

AFAICS, The problem here is that the included filename itself is a result of a preprocessor expression. The current implementation does not support that. Candidate for a github issue.

17:32:08.617 WARN - [C:\Program Files (x86)\Boost\1.49.0\include\boost\preprocessor\iteration\detail\iter\forward1.hpp:47]: cannot find the sources for '# include BOOST_PP_FILENAME_1' Same as above

17:26:50.198 WARN - Error evaluating expression '( ( 0 _WORKAROUND_GUARD + 0 == 0 ) && ( 0 != 0 ) && ( 1 % ( ( ( 0 != ( ( 0x0304 ) - ( 0x0304 ) ) ) ) + 1 ) ) )', assuming 0 We already have an issue for this; not sure whats the root cause here.

17:26:21.272 WARN - [C:\Program Files (x86)\Boost\1.49.0\include\boost\mpl\aux_\include_preprocessed.hpp:37]: cannot find the sources for '# include BOOST_PPSTRINGIZE(boost/mpl/aux/preprocessed/AUX778076_PREPROCESSED_HEADER)' Same as first and second.

17:26:23.661 WARN - Cannot decode the number '0xffffffffffffffffu' falling back to max long (9223372036854775807) instead Yep, we already have an issue for this.

All in all, thats less than I would expect. Last time I played with boost, there were more issues.

— Reply to this email directly or view it on GitHub https://github.com/wenns/sonar-cxx/issues/226#issuecomment-45221332.

[guwirth]

224 / #308 solves one issue

[guwirth]

Some hints from the header file:

// Compiler/library version workaround macro
//
// Usage:
//
//     #if BOOST_WORKAROUND(BOOST_MSVC, < 1300)
//        // workaround for eVC4 and VC6
//        ... // workaround code here
//     #endif
//
// When BOOST_STRICT_CONFIG is defined, expands to 0. Otherwise, the
// first argument must be undefined or expand to a numeric
// value. The above expands to:
//
//     (BOOST_MSVC) != 0 && (BOOST_MSVC) < 1300
//
// When used for workarounds that apply to the latest known version 
// and all earlier versions of a compiler, the following convention 
// should be observed:
//
//     #if BOOST_WORKAROUND(BOOST_MSVC, BOOST_TESTED_AT(1301))
//
// The version number in this case corresponds to the last version in
// which the workaround was known to have been required. When
// BOOST_DETECT_OUTDATED_WORKAROUNDS is not the defined, the macro
// BOOST_TESTED_AT(x) expands to "!= 0", which effectively activates
// the workaround for any version of the compiler. When
// BOOST_DETECT_OUTDATED_WORKAROUNDS is defined, a compiler warning or
// error will be issued if the compiler version exceeds the argument
// to BOOST_TESTED_AT().  This can be used to locate workarounds which
// may be obsoleted by newer versions.

#  define BOOST_WORKAROUND(symbol, test)                \
         ((symbol ## _WORKAROUND_GUARD + 0 == 0) &&     \
         (symbol != 0) && (1 % (( (symbol test) ) + 1)))
//                              ^ ^           ^ ^
// The extra level of parenthesis nesting above, along with the
// BOOST_OPEN_PAREN indirection below, is required to satisfy the
// broken preprocessor in MWCW 8.3 and earlier.
//
// The basic mechanism works as follows:
//      (symbol test) + 1        =>   if (symbol test) then 2 else 1
//      1 % ((symbol test) + 1)  =>   if (symbol test) then 1 else 0
//
// The complication with % is for cooperation with BOOST_TESTED_AT().
// When "test" is BOOST_TESTED_AT(x) and
// BOOST_DETECT_OUTDATED_WORKAROUNDS is #defined,
//
//      symbol test              =>   if (symbol <= x) then 1 else -1
//      (symbol test) + 1        =>   if (symbol <= x) then 2 else 0
//      1 % ((symbol test) + 1)  =>   if (symbol <= x) then 1 else divide-by-zero
//

Some calling examples:

#if !BOOST_WORKAROUND(BOOST_MSVC, < 1300)
#if BOOST_WORKAROUND(BOOST_MSVC, >= 1400)  
#if BOOST_WORKAROUND(__BORLANDC__, <= 0x0550)
#if BOOST_WORKAROUND(__BORLANDC__, BOOST_TESTED_AT(0x564))
#if (BOOST_WORKAROUND(__GNUC__, BOOST_TESTED_AT(4))

Related defines:

#define _MSC_VER 1600
#define BOOST_MSVC _MSC_VER
#define _MSC_VER_WORKAROUND_GUARD 0
#define BOOST_TESTED_AT(value) != ((value)-(value))

[guwirth]

Call of:

#define _MSC_VER_WORKAROUND_GUARD 0
#define _MSC_VER 1600
#define BOOST_MSVC _MSC_VER
... BOOST_WORKAROUND(BOOST_MSVC, < 1300)

#define BOOST_WORKAROUND(symbol, test)                \
         ((symbol ## _WORKAROUND_GUARD + 0 == 0) &&     \
         (symbol != 0) && (1 % (( (symbol test) ) + 1)))

results in:

Error evaluating expression '( ( 1600 _WORKAROUND_GUARD + 0 == 0 ) && ( 1600 != 0 ) && ( 1 % ( ( ( 1600 < 1300 ) ) + 1 ) ) )', assuming 0

Problem seems to be 1600 _WORKAROUND_GUARD should be _MSC_VER_WORKAROUND_GUARD and final 0.

Reduced test case:

#define _MSC_VER_WORKAROUND_GUARD 0
#define _MSC_VER 1600
#define BOOST_MSVC _MSC_VER
#define TEST(symbol) symbol ## _WORKAROUND_GUARD
TEST(BOOST_MSVC); // result should be 0

[guwirth]

Related to #304? I will test it with that fix.

[guwirth]

Problem is not fixed with #304. In testcase above preprocessor resolves macro in the wrong order: BOOST_MSVC => _MSC_VER => 1600 ## _WORKAROUND_GUARD => 1600 _WORKAROUND_GUARD

Test in PreprocessorDirectivesTest.java

  @Test
  public void complex_macro_rescanning() {
    assert (serialize(p.parse(
      "#define _MSC_VER_WORKAROUND_GUARD 0\n"
      + "#define _MSC_VER 1600\n"
      + "#define BOOST_MSVC _MSC_VER\n"
      + "#define TEST(symbol) symbol ## _WORKAROUND_GUARD\n"
      + "TEST(BOOST_MSVC);"))
      .equals("0 ; EOF"));
  }

Preprocessor is a monster :-> I can't fix it.

[guwirth]

During the next days I will try this again (if no one else is working on this currently)

[wenns]

I do not; I had a quick look, realized that the evaluation of ## is sometimes not at the proper point of time but could not spot a solution.

[guwirth]

Problem is replaceParams/newValue. To get this expandMacro is called which is calling handleIdentifiersAndKeywords again. handleIdentifiersAndKeywords calls again expandMacro.

'The arguments have to be fully expanded before expanding the body of the macro' is wrong. Should expand only one level and calling evaluateHashhashOperators then before doing further expansion.

Someone an idea how to fix this?

https://github.com/wenns/sonar-cxx/blob/master/cxx-squid/src/main/java/org/sonar/cxx/preprocessor/CxxPreprocessor.java, line 784

private List<Token> replaceParams(List<Token> body, List<Token> parameters, List<Token> arguments) {
...
// The arguments have to be fully expanded before expanding the body of the macro
String newValue = serialize(expandMacro("", replacement.getValue()));