Replace oro.jar regular expressions with java.regex

[GoogleCodeExporter]

What change would like to see?

Migrating over to Java's built-in regular expressions instead of using oro.jar. 
 Ask Eberhard for 
help.

Why?

So we don't need to continue to include an obsolete regular expression engine.

Would this cause any incompatibilities with previous versions?  If so, how
can these be mitigated?

Original issue reported on code.google.com by dmel...@gmail.com on 6 Jan 2010 at 5:44

[GoogleCodeExporter]

Hi, 

I have been playing with the code a bit, trying to make more suitable for 
creating an
ubuntu package. In order to get rid of the "oro.jar" I had to rewrite
processing.app.preproc.PdePreprocessor.   

The only noticeable change would be that I created a local version of the
scrubComments(String) method. The original code calls 
Sketch.scrubComments(String).
But nothing else uses Sketch.scrubComments(String) so why should this not be 
moved to
the preprocessor class?

Eberhard

Original comment by e.fa...@wayoda.org on 9 Jan 2010 at 1:00

Attachments:

• PdePreprocessor.java

[GoogleCodeExporter]

The file I posted previously had 2 bugs.
1) It reported an IllegalArgumentException instead of a RuntimeException when 
the
user entered unbalanced /* ... */ multiline comments.  
2) It had a problem generating the prototypes when there was a comma in a 
comment
just before the function definition

Both bugs are fixed in the attached file.

Eberhard 

Original comment by e.fa...@wayoda.org on 10 Jan 2010 at 5:48

Attachments:

• PdePreprocessor.java

[GoogleCodeExporter]

Can we put scrubComments() back in Sketch.java?  This will make it easier for 
me to merge any corrections that 
Processing might make to the function, and to provide any changes we make back 
to Processing.  

Also, it would be great to have more information about the tests you've run on 
this.  Either the ones I was using 
before 
<http://code.google.com/p/arduino/source/browse/#svn/branches/junit/app/preproc/
test/data> or 
others.

Original comment by dmel...@gmail.com on 28 Jan 2010 at 8:41

• Changed state: Accepted
• Added labels: Milestone-0019

[GoogleCodeExporter]

Here is a new version of the preprocessor. It now passes all the (valid) tests 
from  
<http://code.google.com/p/arduino/source/browse/#svn/branches/junit/app/preproc/
test/data>.
It fails on t14.cpp, but so does the current implementation! 

I would leave scrubComments() in PdePreprocessor. We want a working 
preprocessor for
Arduino Sketches. If procsessing updates the version in Sketch.scrubComments() 
it
will very likely incompatible with the requirements of processing Arduino 
Sketches. 

Eberhard

Original comment by e.fa...@wayoda.org on 10 Feb 2010 at 11:10

Attachments:

• PdePreprocessor.java

[GoogleCodeExporter]

New Version in reaction to issue 205
http://code.google.com/p/arduino/issues/detail?id=205

Original comment by e.fa...@wayoda.org on 13 Feb 2010 at 4:28

Attachments:

• PdePreprocessor.java

[GoogleCodeExporter]

And another update caused by issue205  
Functions can return references too!

Original comment by e.fa...@wayoda.org on 14 Feb 2010 at 11:18

Attachments:

• PdePreprocessor.java

[GoogleCodeExporter]

Any chance you can submit a more streamlined patch (one that makes the switch 
from the old oro.jar regular 
expressions to java.regex ones, but without other modifications to the code)?  
The other changes may be useful, 
but make it more difficult to understand what's going on here.  Maybe we can 
implement them separately?

Original comment by dmel...@gmail.com on 19 Feb 2010 at 8:31

[GoogleCodeExporter]

Original comment by dmel...@gmail.com on 11 Aug 2010 at 6:09

• Removed labels: Milestone-0019

[GoogleCodeExporter]

@dmellis, you can get the patch in attachment, some notes:

1) I can confirm that the patch passes all tests in the old repository (except 
t14.cpp but i dont know if it's valid).

2) Eberhard make some changes in regexes related to comments:

    // single and multi-line comment
    // p += "|" + "(//\\s*?$)|(/\\*\\s*?\\*/)";             <- E. version
    p += "|(//.*?$)|(/\\*[^*]*(?:\\*(?!/)[^*]*)*\\*/)";     <- actual

but seems that the actual version is the correct one, since the Eberhard one 
never matches comments.

3) Eberhard made another change in prototype:

"[\\w\\[\\]\\*]+\\s+[&\\[\\]\\*\\w\\s]+\\([&,\\[\\]\\*\\w\\s]*\\)(?=\\s*\\{)"   
  <- E. version
"[\\w\\[\\]\\*]+\\s+[\\[\\]\\*\\w\\s]+\\([,\\[\\]\\*\\w\\s]*\\)(?=\\s*\\{)"     
  <- actual

he added the two amps that matches argument passed by reference like:

void func(MyStruct &var);

this one seems correct to me and I keeped it.

4) I've not tested the method firstStatement(...). 
It seems a recently added method that wasn't ported yet. I tried an 
implementation but is not tested and can (probably) fail. Can you give me some 
tests to check that?

Original comment by c.mag...@bug.st on 1 Oct 2010 at 3:07

Attachments:

• regexp.patch

[GoogleCodeExporter]

Ok, use the one attached here, this patch solves also the firstStatement(...) 
issue.
I checked with some examples (using the old method and the new) and both gives 
the same results.

Original comment by c.mag...@bug.st on 1 Oct 2010 at 4:13

Attachments:

• regexp.patch

[GoogleCodeExporter]

Christian: looks good, I applied it: 
http://github.com/arduino/Arduino/commit/fa4d0582970e50c574ecfdcdbcfeae3754ca209
5

Original comment by dmel...@gmail.com on 4 Oct 2010 at 12:09

• Changed state: Fixed
• Added labels: Milestone-0022