Preprocessor macros

[EliotVU]

The usage of macros can lead to severe parsing and indexing issues, like the following code:

return ( `TimeSince( LastRenderTime ) < 0.3 );

The usage of `TimeSince here will tell the lexer that this is a PP command, when this is occurred the input will be thrown out thus leaving the parser with:

return ( < 0.3 );

Which leads to even bigger issues down the road, so as the end block of a method not being terminated.

One workaround for this issue could be to safely parse binary operators that are missing an expression on the left or right, however due the nature of UnrealScript allowance of custom defined pre and post operators, "< 0.3" could in theory be a post operator! Thus it cannot be assumed to be an incomplete binary operator (albeit it would be totally unexpected for "<" symbol, that cannot be said of other symbols).

The proper solution would be not to skip any PP commands and instead fully parse and evaluate any macro prior to parsing the document, this is however not an easy task!

[Xymanek]

Status of marcos in 0.4.3:

Parsing still fails. Sometimes it causes just a small error:

Sometimes a much bigger problem: The "problems" window is full of (ANTLR Node Error) ;

[EliotVU]

Yeah, macros with params are not yet supported, neither are macro symbols being included yet from the global.uci or `include calls.

[tuokri]

I was thinking, would it be possible to use the output from make -intermediate to improve the plugin's macro parsing? Whenever a macro with params occurs, fetch the correct code from the intermediate output, instead of just throwing out the macro. Though I have no idea whether it's possible to keep the source code and intermediate output in sync to be able to programmatically track which parts of the intermediate output are preprocessed macros and which parts correspond to the unprocessed source directly.

[EliotVU]

@tuokri Yes, or even just displace the workspace folders with macro code with the intermediate code; The only problem with this solution would be for modders that need to mod the stock scripts.

[EliotVU]

I have been working on and off on a new approach to tackle this issue in another branch, but nothing complete yet (ANTLR's approach really likes to interfere with most solutions)

[EliotVU]

I have worked through the road blocking issues I was having with the re-worked preprocessor, see commit 4aefe32d3e22b30adacf96d40475743304e61316

It can now preprocess:

• Any include eveninclude within any `include including all its macro and non-macro tokens!
• Macros with arguments `Log(text...)
• Concatenated macros `{prefix}Identifier

However this is not yet production ready, there's still work to be done in regards to:

• Counter, SetCounter and GetCounter macros.
• Syntax highlighting, proper hoverinfo and goto definition handling.
• Incomplete typed macros
• Macro inheritance and global macros relative to the 'package' directory.
• Cannot parse `{endif}
• Non-macro code between a if andendif block are always included when they shouldn't be.