Minus sign and number constant recognized as a single token

[grzegorz8]

Since this commit https://github.com/shevek/jcpp/commit/ca42036955f2589d034a688267eb81c5da2c824d the occurrence of minus sign followed by a number constant is returned by preprocessor as a single token. I wonder why the minus sign is not considered to be a separate token (as it was previously)? Is it required by some preprocessor specification?

[shevek]

Yes, that was deliberate. I didn't actually look up C99, and I was trying to be "helpful", per some commits I saw elsewhere.

I have now looked up C99 section 6.4.4.1, and you are right, the - sign is not a part of the number.

I will fix this. Thank you for bringing my attention to it.

Interestingly, previously I considered jcpp only as an accurate text processor, but it's clear from the community response that the accuracy of the tokenization is also very important to a lot of people, so I will be focusing increasingly on responding to bug reports on that topic.

However, it's also my intention that it be a useful tokenizer for JavaScript, Groovy and so forth where that does not conflict with C99, hence allowing multichar single quoted strings (per an early bug report).

[shevek]

I'll also take any other tests for numerics, especially ones that illustrate how lazily falling back to Java's Double parsing is incorrect for C99. I'm happy to write floating point routines.

[shevek]

I have patch and tests for this. They will land Very Soon.

[shevek]

https://github.com/shevek/jcpp/commit/ec176d2a623f274ad8b0d7962b88a2f7a67446ca