java.lang.StackOverflowError when preprocessing broken sketch

[GoogleCodeExporter]

What steps will reproduce the problem?
1. Open the attached testcase.ino sketch.
2. Try to compile it. (Yes I know the code is gibberish)

What is the expected output? What do you see instead?
Expected:
For the original program, successful compilation(it compiles just fine if the 
Serial.println("test line is removed).
For the testcase, any error message originating from the compiler about broken 
source code.
Actual:
java.lang.StackOverflowError exception in some RegEx code, see 
errormsg-from-compiling-testcase.txt for the full output.

What version of the Arduino software are you using? On what operating
system?  Which Arduino board are you using?
Arduino 1.0 and latest github version.
Win7 and Ubuntu linux.
Arduino Uno.

Please provide any additional information below.
The things that are commented out can not be deleted, then the process gets to 
the compilation stage.

Original issue reported on code.google.com by lars.j.n...@gmail.com on 15 Mar 2012 at 5:11

Attachments:

• testcase.ino
• errormsg-from-compiling-testcase.txt

[GoogleCodeExporter]

I first ran this through the auto-formatter and it had a problem running out of 
heap.
There are several while() loops that don't terminate if the reader encounters 
EOF.  Here's a patch that handles that problem.

Original comment by ericzun...@gmail.com on 11 Jul 2012 at 11:16

Attachments:

• auto-format-bug.patch

[GoogleCodeExporter]

This patch fixes the stack overflow and refactors the PdePreprocessor.strip() 
method.  I tried a few tests and they seem to work.  You might just want to 
take the change to the regular expression - I didn't try backporting it though.

Essentially, I don't think the regular expression for double quoted string is 
correct

    // double-quoted string
    p += "|(\"(?:[^\"\\\\]|\\\\.)*\")";

inside the brackets, this is a class of characters, so [^\\\\] is redundant.  I 
changed it to  "((\"\")|\"((\\\\.)|.)+\")"

Original comment by ericzun...@gmail.com on 12 Jul 2012 at 3:16

Attachments:

• issue-859.patch

[GoogleCodeExporter]

I committed the EOF checks in the auto-format code: 
https://github.com/arduino/Arduino/commit/d903d81d609f2fbd97bfe37c6af05054f43874
93

One question, what is the purpose of this code: if (++count % 100000 == 0) {  
System.err.println("Stuck: " + count); } ?  Do we need to keep it in there?

I'm confused about the changes to the strip method.  I tried compiling the test 
case in the original bug report and it compiled fine with the latest Arduino 
code (in GitHub) on Mac.  Is there another test case or operating system I 
should be trying?  You mentioned that I might just want to update one of the 
regular expressions; can you post a patch just for that?  I was getting a bit 
lost in the changes (and all the regexps) for the previous patch.

Original comment by dmel...@gmail.com on 15 Jul 2012 at 1:35

• Added labels: Component-IDE

[GoogleCodeExporter]

The loop in the auto-format can be discarded, I accidentally left it in (was 
using it to set a breakpoint to debug the problem.)

To reproduce the problem with the testcase.ino file in the initial bug report, 
I opened the file, then hit the 'Verify' action on the toolbar and got a stack 
trace.  Possibly the difference in reproducing it is due to the version of the 
JDK since I'm sure regular expressions are implemented differently between 
OpenJDK and the Oracle JDK.

$ java -showversion
java version "1.6.0_24"
OpenJDK Runtime Environment (IcedTea6 1.11.1) (6b24-1.11.1-4ubuntu3)
OpenJDK 64-Bit Server VM (build 20.0-b12, mixed mode)

Attached is the one-line fix

Original comment by ericzun...@gmail.com on 16 Jul 2012 at 12:11

Attachments:

• fix-other-way.patch

[GoogleCodeExporter]

Thanks for the update.  I removed the leftover code: 
https://github.com/arduino/Arduino/commit/a01657b31241e08acbc03768fb4a13c0349597
ae

These regular expression tweaks make me realize we really need tests for the 
pre-processing (and auto-format) code.  In particular, won't your change assume 
that the string continues until the last double-quote (") on a line?  For 
example, in:

void foo() {Serial.print("foo");} void bar() {Serial.print("bar");}

wouldn't the updated regular expression match: "foo");} void bar() 
{Serial.print("bar" ?  This would mean, for example, that we'd miss the 
declaration of bar() and not generate a prototype for it.  Or am I 
misunderstanding how the regex works?

Original comment by dmel...@gmail.com on 19 Jul 2012 at 2:55

[GoogleCodeExporter]

Please give a try to the IDEs linked at the bottom of this email on the dev list
https://groups.google.com/a/arduino.cc/forum/#!msg/developers/4X2T3rCxXWM/YNJl6P
ZuDucJ
We're testing a new preprocessor and it compiles your sketch just fine.

Original comment by federico...@gmail.com on 28 Jan 2015 at 4:13

[GoogleCodeExporter]

New preprocessor tracked at https://github.com/arduino/Arduino/pull/2636. 
Builds for testing it are available

Original comment by federico...@gmail.com on 13 Feb 2015 at 6:49

[GoogleCodeExporter]

Original comment by federico...@gmail.com on 26 May 2015 at 3:26

• Changed state: Fixed