Preprocessor inserts spaces in macro expansions

[llvmbot]

Bugzilla Link	12035
Version	trunk
OS	MacOS X
Reporter	LLVM Bugzilla Contributor
CC	@efriedma-quic

Extended Description

clang 3.1 which is shipping with Xcode 4.3 ("based on LLVM 3.1svn") seems to insert extra spaces in macro expansions.

A test case:

$ cat foo.h

define VALUE 2012-02-19

Output: VALUE

$ clang -E -P foo.h

Output: 2012 -02 -19

$ gcc -E -P foo.h

Output: 2012-02-19

$ clang --version Apple clang version 3.1 (tags/Apple/clang-318.0.45) (based on LLVM 3.1svn) Target: x86_64-apple-darwin11.3.0 Thread model: posix

Notes:

• Apple Radar #​10883862

[llvmbot]

As of Xcode 7.2, the -traditional flag is not working as described here:

https://developer.apple.com/library/mac/#technotes/tn2175/_index.html

I'm trying to construct a version number for my Info.plist as described in the last example, and it is putting spaces before and after each period.

[llvmbot]

This seems fixed now, which I am very glad to see.

This was a total bug. In cpp, the replacement is EXACTLY the same. Other than something like substituting "one space" for "multiple spaces", cpp does not "clean up" or "tidy" code. It was totally wrong to insert a space willy-nilly like it was previously doing, arbitrarily changing "2012-02-19" to "2012 -02 -19".

Anyway, here is the current (correct) behavior:

$ clang --version Ubuntu clang version 3.4-1ubuntu3 (tags/RELEASE_34/final) (based on LLVM 3.4) Target: x86_64-pc-linux-gnu Thread model: posix

$ cat clang-2.h

define DATE 2012-02-19 X

DATE

$ gcc -E -P clang-2.h 2012-02-19 X

$ clang -E -P clang-2.h

2012-02-19 X

Thanks very much to whoever fixed this.

[llvmbot]

clang's preprocessor is a C preprocessor; the primary point is that the output parses into the same C tokens as the input. We don't make any guarantees about preserving whitespace or the lack of whitespace between tokens in preprocessed source.

Just to provide some context to how this issue was discovered: clang's preprocessor is used by Xcode 4.3 to preprocess Info.plist files. Some Xcode users use this functionality to define macros in Info.plist file which then expand to URL values. In fact, Apple acknowledges this use case by mentioning it in their documentation: https://developer.apple.com/library/mac/#technotes/tn2175/_index.html

So while current functionality of clang preprocessor might be correct from C preprocessor point of view, it currently causes problems in those use cases in which it's widely used as Info.plist preprocessor and as promoted by Apple. From that perspective this might be more of Apple's documentation issue (updating TN2175 to take clang preprocessor's implementation details into account), which rdar://10883862 will hopefully help to address.

[efriedma-quic]

clang's preprocessor is a C preprocessor; the primary point is that the output parses into the same C tokens as the input. We don't make any guarantees about preserving whitespace or the lack of whitespace between tokens in preprocessed source.

That said, clang could probably do better in this particular case.