Error in new XWiki 2.0 parser


I found a serious error.  The lexical state is not switched after a heading
that has content additional content on the same line. I cannot believe that
this error slipped through as I thought I was beeing particularly careful
to inspect all cases where the lexical state must be switched.  :(

Here is a unit test:

        test("= header = paragraph\n\nparagraph\n\n", 
             "<h1>header</h1>\n"+
             "<p> paragraph</p>\n"+
             "<p>paragraph</p>");

Here is a patch:

Index: src/main/java/org/wikimodel/wem/xwiki/xwiki20/javacc/XWikiScanner.jj
===================================================================
--- src/main/java/org/wikimodel/wem/xwiki/xwiki20/javacc/XWikiScanner.jj
(revision 446)
+++ src/main/java/org/wikimodel/wem/xwiki/xwiki20/javacc/XWikiScanner.jj
(arbetskopia)
@@ -187,7 +187,7 @@
      *
      * Except for state transitions to BEGINNING_OF_LINE and to
      * special states, we need to control the state transitions in
-     * java code.  For this control the state we keep three variables:
+     * java code.  For this control the state we keep two variables:
      *
      * * preceedingSpecialState
      * * stateStack
@@ -301,7 +301,8 @@
 <HEADER_CONTEXT> TOKEN:
 {
       <HEADER_END: (<SPACE>)*  ("=")+ <NEW_LINE> > {clearBlockState();} :
BEGINNING_OF_LINE
-    | <HEADER_END_INLINE: (<SPACE>)* ("=")+> { 
+    | <HEADER_END_INLINE: (<SPACE>)* ("=")+> {
+          clearBlockState();
           matchedToken.kind = HEADER_END;
       } : INLINE
     | <NEW_HEADER_BEGIN: <NEW_LINE> <HEADER_BEGIN_PATTERN>> {
matchedToken.kind = HEADER_BEGIN; }

Original issue reported on code.google.com by AndreasZ...@gmail.com on 19 Mar 2010 at 12:03

Merged into: #169

quetzai / wikimodel

Error in new XWiki 2.0 parser #176