Fails with JavaCC 6.1.3

sebastic commented 2 years ago

josm fails to build with JavaCC 6:

    [javac] /<<PKGBUILDDIR>>/src/ch/poole/openinghoursparser/OpeningHoursParserTokenManager.java:1182: error: incompatible types: possible lossy conversion from int to char
    [javac]      throw new TokenMgrError(EOFSeen, curLexState, error_line, error_column, error_after, curChar, TokenMgrError.LEXICAL_ERROR);
    [javac]                                                                                           ^

This can be fixed by building with the javacc5 package instead of javacc

ant task used to generate the code:

   <target name="javacc-openinghoursparser" depends="init" unless="javacc.notRequired">
       <mkdir dir="${src.dir}/ch/poole/openinghoursparser/parsergen"/>
       <exec append="false" executable="javacc" failifexecutionfails="true">
           <arg value="-DEBUG_PARSER=false"/>
           <arg value="-DEBUG_TOKEN_MANAGER=false"/>
           <arg value="-JDK_VERSION=1.${java.lang.version}"/>
           <arg value="-GRAMMAR_ENCODING=UTF-8"/>
           <arg value="-UNICODE_INPUT=true"/>
           <arg value="-OUTPUT_DIRECTORY=${src.dir}/ch/poole/openinghoursparser"/>
           <arg value="${src.dir}/ch/poole/openinghoursparser/OpeningHoursParser.jj"/>
       </exec>
   </target>

simonpoole commented 2 years ago

Not really a bug as there was/is no expectation one way or the other that it would build with JavaCC 6. I'll have a look at it ASAP.

simonpoole commented 2 years ago

@sebastic which javacc version were you actually using? I can't find a published 6.0.0 artifact, and, fun, with 6.1.2 things compile (and pass tests) just fine.

sebastic commented 2 years ago

6.1.3: https://tracker.debian.org/pkg/javacc

The javacc Debian package used the release_6_1_3 tag to get the sources.

simonpoole commented 2 years ago

Sigh, see https://github.com/javacc/javacc/releases/tag/release_6_1_3 never actually published and not backwards compatible with prior 6.1 versions it seems.

Don't think this is cleanly fixable before Debian switches to an actually released version of JavaCC.

PS: even if 6.1.3 was an actual release, I would suspect that including a non-backwards compatible patch release would be a big nono in any case. PPS: the parser compiles just fine with the 7.0.x releases too, so it is just 6.1.3 that is the odd one out.

sebastic commented 2 years ago

The javacc package in Debian has been updated to 7.0.11 which has the same issue:

javacc:
    [mkdir] Created dir: /build/josm-0.0.svn18427+dfsg/src/org/openstreetmap/josm/gui/mappaint/mapcss/parsergen
     [exec] Current OS is Linux
     [exec] Executing 'javacc' with arguments:
     [exec] '-DEBUG_PARSER=false'
     [exec] '-DEBUG_TOKEN_MANAGER=false'
     [exec] '-JDK_VERSION=1.8'
     [exec] '-GRAMMAR_ENCODING=UTF-8'
     [exec] '-UNICODE_INPUT=true'
     [exec] '-OUTPUT_DIRECTORY=/build/josm-0.0.svn18427+dfsg/src/org/openstreetmap/josm/gui/mappaint/mapcss/parsergen'
     [exec] '/build/josm-0.0.svn18427+dfsg/src/org/openstreetmap/josm/gui/mappaint/mapcss/MapCSSParser.jj'
     [exec] 
     [exec] The ' characters around the executable and arguments are
     [exec] not part of the command.
Execute:Java13CommandLauncher: Executing 'javacc' with arguments:
'-DEBUG_PARSER=false'
'-DEBUG_TOKEN_MANAGER=false'
'-JDK_VERSION=1.8'
'-GRAMMAR_ENCODING=UTF-8'
'-UNICODE_INPUT=true'
'-OUTPUT_DIRECTORY=/build/josm-0.0.svn18427+dfsg/src/org/openstreetmap/josm/gui/mappaint/mapcss/parsergen'
'/build/josm-0.0.svn18427+dfsg/src/org/openstreetmap/josm/gui/mappaint/mapcss/MapCSSParser.jj'

The ' characters around the executable and arguments are
not part of the command.
     [exec] Java Compiler Compiler Version 7.0.11 (Parser Generator)
     [exec] (type "javacc" with no arguments for help)
     [exec] Warning: Line 4, Column 3: Command line setting of "OUTPUT_DIRECTORY" modifies option value in file.
     [exec] Reading from file /build/josm-0.0.svn18427+dfsg/src/org/openstreetmap/josm/gui/mappaint/mapcss/MapCSSParser.jj . . .
     [exec] Note: UNICODE_INPUT option is specified. Please make sure you create the parser/lexer using a Reader with the correct character encoding.
     [exec] File "TokenMgrError.java" does not exist.  Will create one.
     [exec] File "ParseException.java" does not exist.  Will create one.
     [exec] File "Token.java" does not exist.  Will create one.
     [exec] File "SimpleCharStream.java" does not exist.  Will create one.
     [exec] Parser generated with 0 errors and 1 warnings.

javacc-openinghoursparser:
    [mkdir] Created dir: /build/josm-0.0.svn18427+dfsg/src/ch/poole/openinghoursparser/parsergen
     [exec] Current OS is Linux
     [exec] Executing 'javacc' with arguments:
     [exec] '-DEBUG_PARSER=false'
     [exec] '-DEBUG_TOKEN_MANAGER=false'
     [exec] '-JDK_VERSION=1.8'
     [exec] '-GRAMMAR_ENCODING=UTF-8'
     [exec] '-UNICODE_INPUT=true'
     [exec] '-OUTPUT_DIRECTORY=/build/josm-0.0.svn18427+dfsg/src/ch/poole/openinghoursparser'
     [exec] '/build/josm-0.0.svn18427+dfsg/src/ch/poole/openinghoursparser/OpeningHoursParser.jj'
     [exec] 
     [exec] The ' characters around the executable and arguments are
     [exec] not part of the command.
Execute:Java13CommandLauncher: Executing 'javacc' with arguments:
'-DEBUG_PARSER=false'
'-DEBUG_TOKEN_MANAGER=false'
'-JDK_VERSION=1.8'
'-GRAMMAR_ENCODING=UTF-8'
'-UNICODE_INPUT=true'
'-OUTPUT_DIRECTORY=/build/josm-0.0.svn18427+dfsg/src/ch/poole/openinghoursparser'
'/build/josm-0.0.svn18427+dfsg/src/ch/poole/openinghoursparser/OpeningHoursParser.jj'

The ' characters around the executable and arguments are
not part of the command.
     [exec] Java Compiler Compiler Version 7.0.11 (Parser Generator)
     [exec] (type "javacc" with no arguments for help)
     [exec] Reading from file /build/josm-0.0.svn18427+dfsg/src/ch/poole/openinghoursparser/OpeningHoursParser.jj . . .
     [exec] Note: UNICODE_INPUT option is specified. Please make sure you create the parser/lexer using a Reader with the correct character encoding.
     [exec] Warning: Choice conflict involving two expansions at
     [exec]          line 552, column 5 and line 560, column 5 respectively.
     [exec]          A common prefix is: <NUMBER>
     [exec]          Consider using a lookahead of 2 for earlier expansion.
     [exec] Warning: Choice conflict in (...)* construct at line 658, column 5.
     [exec]          Expansion nested within construct and expansion following construct
     [exec]          have common prefixes, one of which is: "," <NUMBER>
     [exec]          Consider using a lookahead of 3 or more for nested expansion.
     [exec] Warning: Choice conflict in [...] construct at line 781, column 3.
     [exec]          Expansion nested within construct and expansion following construct
     [exec]          have common prefixes, one of which is: <HYPHEN> <NUMBER>
     [exec]          Consider using a lookahead of 3 or more for nested expansion.
     [exec] Warning: Choice conflict in (...)* construct at line 818, column 3.
     [exec]          Expansion nested within construct and expansion following construct
     [exec]          have common prefixes, one of which is: "," <WEEKDAY>
     [exec]          Consider using a lookahead of 3 or more for nested expansion.
     [exec] Warning: Choice conflict in (...)* construct at line 871, column 3.
     [exec]          Expansion nested within construct and expansion following construct
     [exec]          have common prefixes, one of which is: "," <HOLIDAYS>
     [exec]          Consider using a lookahead of 3 or more for nested expansion.
     [exec] Warning: Choice conflict in [...] construct at line 1004, column 3.
     [exec]          Expansion nested within construct and expansion following construct
     [exec]          have common prefixes, one of which is: <NUMBER>
     [exec]          Consider using a lookahead of 2 for nested expansion.
     [exec] Warning: Choice conflict involving two expansions at
     [exec]          line 1011, column 5 and line 1092, column 5 respectively.
     [exec]          A common prefix is: <MONTH>
     [exec]          Consider using a lookahead of 2 for earlier expansion.
     [exec] Warning: Choice conflict involving two expansions at
     [exec]          line 1015, column 11 and line 1032, column 11 respectively.
     [exec]          A common prefix is: <MONTH>
     [exec]          Consider using a lookahead of 2 for earlier expansion.
     [exec] Warning: Choice conflict in [...] construct at line 1066, column 7.
     [exec]          Expansion nested within construct and expansion following construct
     [exec]          have common prefixes, one of which is: <HYPHEN>
     [exec]          Consider using a lookahead of 2 for nested expansion.
     [exec] Warning: Choice conflict in [...] construct at line 1077, column 7.
     [exec]          Expansion nested within construct and expansion following construct
     [exec]          have common prefixes, one of which is: <HYPHEN> <NUMBER>
     [exec]          Consider using a lookahead of 3 or more for nested expansion.
     [exec] Warning: Choice conflict in (...)* construct at line 1208, column 3.
     [exec]          Expansion nested within construct and expansion following construct
     [exec]          have common prefixes, one of which is: "," <NUMBER>
     [exec]          Consider using a lookahead of 3 or more for nested expansion.
     [exec] Warning: Choice conflict in (...)* construct at line 1268, column 3.
     [exec]          Expansion nested within construct and expansion following construct
     [exec]          have common prefixes, one of which is: "," <NUMBER>
     [exec]          Consider using a lookahead of 3 or more for nested expansion.
     [exec] Warning: Choice conflict in [...] construct at line 1333, column 7.
     [exec]          Expansion nested within construct and expansion following construct
     [exec]          have common prefixes, one of which is: <NUMBER> <HYPHEN>
     [exec]          Consider using a lookahead of 3 or more for nested expansion.
     [exec] Warning: Choice conflict in [...] construct at line 1339, column 7.
     [exec]          Expansion nested within construct and expansion following construct
     [exec]          have common prefixes, one of which is: <NUMBER> <HYPHEN>
     [exec]          Consider using a lookahead of 3 or more for nested expansion.
     [exec] Warning: Choice conflict in [...] construct at line 1342, column 7.
     [exec]          Expansion nested within construct and expansion following construct
     [exec]          have common prefixes, one of which is: ":"
     [exec]          Consider using a lookahead of 2 for nested expansion.
     [exec] Warning: Choice conflict involving two expansions at
     [exec]          line 1380, column 9 and line 1405, column 9 respectively.
     [exec]          A common prefix is: <WEEKDAY> <HYPHEN>
     [exec]          Consider using a lookahead of 3 or more for earlier expansion.
     [exec] Warning: Choice conflict in [...] construct at line 1385, column 11.
     [exec]          Expansion nested within construct and expansion following construct
     [exec]          have common prefixes, one of which is: "," <WEEKDAY>
     [exec]          Consider using a lookahead of 3 or more for nested expansion.
     [exec] Warning: Choice conflict in [...] construct at line 1409, column 11.
     [exec]          Expansion nested within construct and expansion following construct
     [exec]          have common prefixes, one of which is: "," <WEEKDAY>
     [exec]          Consider using a lookahead of 3 or more for nested expansion.
     [exec] Parser generated with 0 errors and 28 warnings.
     [exec] Warning: TokenMgrError.java: File is obsolete.  Please rename or delete this file so that a new one can be generated for you.
     [exec] Warning: TokenMgrError.java file   version: 5.0 javacc version: 7.0
     [exec] Warning: ParseException.java: File is obsolete.  Please rename or delete this file so that a new one can be generated for you.
     [exec] Warning: ParseException.java file   version: 5.0 javacc version: 7.0
     [exec] Warning: ParseException.java: Generated using incompatible options. Please rename or delete this file so that a new one can be generated for you.
     [exec] Warning: Token.java: File is obsolete.  Please rename or delete this file so that a new one can be generated for you.
     [exec] Warning: Token.java file   version: 5.0 javacc version: 7.0
     [exec] Warning: Token.java: Generated using incompatible options. Please rename or delete this file so that a new one can be generated for you.
     [exec] Warning: SimpleCharStream.java: File is obsolete.  Please rename or delete this file so that a new one can be generated for you.
     [exec] Warning: SimpleCharStream.java file   version: 5.0 javacc version: 7.0
[...]
    [javac] /build/josm-0.0.svn18427+dfsg/src/ch/poole/openinghoursparser/OpeningHoursParserTokenManager.java:1183: error: incompatible types: possible lossy conversion from int to char
    [javac]      throw new TokenMgrError(EOFSeen, curLexState, error_line, error_column, error_after, curChar, TokenMgrError.LEXICAL_ERROR);
    [javac]                                                                                           ^

It works when the generated files that are included in the source JAR are removed from the JOSM source tree prior to the build:

javacc-openinghoursparser:
    [mkdir] Created dir: /build/josm-0.0.svn18427+dfsg/src/ch/poole/openinghoursparser/parsergen
     [exec] Current OS is Linux
     [exec] Executing 'javacc' with arguments:
     [exec] '-DEBUG_PARSER=false'
     [exec] '-DEBUG_TOKEN_MANAGER=false'
     [exec] '-JDK_VERSION=1.8'
     [exec] '-GRAMMAR_ENCODING=UTF-8'
     [exec] '-UNICODE_INPUT=true'
     [exec] '-OUTPUT_DIRECTORY=/build/josm-0.0.svn18427+dfsg/src/ch/poole/openinghoursparser'
     [exec] '/build/josm-0.0.svn18427+dfsg/src/ch/poole/openinghoursparser/OpeningHoursParser.jj'
     [exec] 
     [exec] The ' characters around the executable and arguments are
     [exec] not part of the command.
Execute:Java13CommandLauncher: Executing 'javacc' with arguments:
'-DEBUG_PARSER=false'
'-DEBUG_TOKEN_MANAGER=false'
'-JDK_VERSION=1.8'
'-GRAMMAR_ENCODING=UTF-8'
'-UNICODE_INPUT=true'
'-OUTPUT_DIRECTORY=/build/josm-0.0.svn18427+dfsg/src/ch/poole/openinghoursparser'
'/build/josm-0.0.svn18427+dfsg/src/ch/poole/openinghoursparser/OpeningHoursParser.jj'

The ' characters around the executable and arguments are
not part of the command.
     [exec] Java Compiler Compiler Version 7.0.11 (Parser Generator)
     [exec] (type "javacc" with no arguments for help)
     [exec] Reading from file /build/josm-0.0.svn18427+dfsg/src/ch/poole/openinghoursparser/OpeningHoursParser.jj . . .
     [exec] Note: UNICODE_INPUT option is specified. Please make sure you create the parser/lexer using a Reader with the correct character encoding.
     [exec] Warning: Choice conflict involving two expansions at
     [exec]          line 552, column 5 and line 560, column 5 respectively.
     [exec]          A common prefix is: <NUMBER>
     [exec]          Consider using a lookahead of 2 for earlier expansion.
     [exec] Warning: Choice conflict in (...)* construct at line 658, column 5.
     [exec]          Expansion nested within construct and expansion following construct
     [exec]          have common prefixes, one of which is: "," <NUMBER>
     [exec]          Consider using a lookahead of 3 or more for nested expansion.
     [exec] Warning: Choice conflict in [...] construct at line 781, column 3.
     [exec]          Expansion nested within construct and expansion following construct
     [exec]          have common prefixes, one of which is: <HYPHEN> <NUMBER>
     [exec]          Consider using a lookahead of 3 or more for nested expansion.
     [exec] Warning: Choice conflict in (...)* construct at line 818, column 3.
     [exec]          Expansion nested within construct and expansion following construct
     [exec]          have common prefixes, one of which is: "," <WEEKDAY>
     [exec]          Consider using a lookahead of 3 or more for nested expansion.
     [exec] Warning: Choice conflict in (...)* construct at line 871, column 3.
     [exec]          Expansion nested within construct and expansion following construct
     [exec]          have common prefixes, one of which is: "," <HOLIDAYS>
     [exec]          Consider using a lookahead of 3 or more for nested expansion.
     [exec] Warning: Choice conflict in [...] construct at line 1004, column 3.
     [exec]          Expansion nested within construct and expansion following construct
     [exec]          have common prefixes, one of which is: <NUMBER>
     [exec]          Consider using a lookahead of 2 for nested expansion.
     [exec] Warning: Choice conflict involving two expansions at
     [exec]          line 1011, column 5 and line 1092, column 5 respectively.
     [exec]          A common prefix is: <MONTH>
     [exec]          Consider using a lookahead of 2 for earlier expansion.
     [exec] Warning: Choice conflict involving two expansions at
     [exec]          line 1015, column 11 and line 1032, column 11 respectively.
     [exec]          A common prefix is: <MONTH>
     [exec]          Consider using a lookahead of 2 for earlier expansion.
     [exec] Warning: Choice conflict in [...] construct at line 1066, column 7.
     [exec]          Expansion nested within construct and expansion following construct
     [exec]          have common prefixes, one of which is: <HYPHEN>
     [exec]          Consider using a lookahead of 2 for nested expansion.
     [exec] Warning: Choice conflict in [...] construct at line 1077, column 7.
     [exec]          Expansion nested within construct and expansion following construct
     [exec]          have common prefixes, one of which is: <HYPHEN> <NUMBER>
     [exec]          Consider using a lookahead of 3 or more for nested expansion.
     [exec] Warning: Choice conflict in (...)* construct at line 1208, column 3.
     [exec]          Expansion nested within construct and expansion following construct
     [exec]          have common prefixes, one of which is: "," <NUMBER>
     [exec]          Consider using a lookahead of 3 or more for nested expansion.
     [exec] Warning: Choice conflict in (...)* construct at line 1268, column 3.
     [exec]          Expansion nested within construct and expansion following construct
     [exec]          have common prefixes, one of which is: "," <NUMBER>
     [exec]          Consider using a lookahead of 3 or more for nested expansion.
     [exec] Warning: Choice conflict in [...] construct at line 1333, column 7.
     [exec]          Expansion nested within construct and expansion following construct
     [exec]          have common prefixes, one of which is: <NUMBER> <HYPHEN>
     [exec]          Consider using a lookahead of 3 or more for nested expansion.
     [exec] Warning: Choice conflict in [...] construct at line 1339, column 7.
     [exec]          Expansion nested within construct and expansion following construct
     [exec]          have common prefixes, one of which is: <NUMBER> <HYPHEN>
     [exec]          Consider using a lookahead of 3 or more for nested expansion.
     [exec] Warning: Choice conflict in [...] construct at line 1342, column 7.
     [exec]          Expansion nested within construct and expansion following construct
     [exec]          have common prefixes, one of which is: ":"
     [exec]          Consider using a lookahead of 2 for nested expansion.
     [exec] Warning: Choice conflict involving two expansions at
     [exec]          line 1380, column 9 and line 1405, column 9 respectively.
     [exec]          A common prefix is: <WEEKDAY> <HYPHEN>
     [exec]          Consider using a lookahead of 3 or more for earlier expansion.
     [exec] Warning: Choice conflict in [...] construct at line 1385, column 11.
     [exec]          Expansion nested within construct and expansion following construct
     [exec]          have common prefixes, one of which is: "," <WEEKDAY>
     [exec]          Consider using a lookahead of 3 or more for nested expansion.
     [exec] Warning: Choice conflict in [...] construct at line 1409, column 11.
     [exec]          Expansion nested within construct and expansion following construct
     [exec]          have common prefixes, one of which is: "," <WEEKDAY>
     [exec]          Consider using a lookahead of 3 or more for nested expansion.
     [exec] File "TokenMgrError.java" does not exist.  Will create one.
     [exec] File "ParseException.java" does not exist.  Will create one.
     [exec] File "Token.java" does not exist.  Will create one.
     [exec] File "SimpleCharStream.java" does not exist.  Will create one.
     [exec] Parser generated with 0 errors and 18 warnings.

simonpoole / OpeningHoursParser

Fails with JavaCC 6.1.3 #72