Allow IToken interface as second parameter of pattern definition

[GoogleCodeExporter]

It would be great if you could use an Enum as the second parameter of
@Lex() when defining a pattern. So instead of

 @Lex (pattern = "[;]", token = "SEMICOLON"),

you could write

 @Lex (pattern = "[;]", token = Token.SEMICOLON),

Since Enums can't be extended, I suggest to define a marker interface
"IToken" which token enums must implement and allow to use anything which
implements IToken as second parameter.

Original issue reported on code.google.com by adigu...@gmail.com on 1 Dec 2008 at 9:46

[GoogleCodeExporter]

There are several problems.  For terminals such as ':', ';' etc that are not 
defined
in token enum, such approach would not work.  Also, because token enum is parsed
separately (APT basically treat them as separate classes), and because we are 
running
APT in nocompile mode, such we can't check the integrity anyways.  Of course, 
being
able to specify token directly would prevent spelling errors.

So I don't know.  Need to think about it more.

Original comment by superdup...@gmail.com on 1 Dec 2008 at 12:17

• Changed state: Accepted
• Added labels: Priority-Low, Type-Enhancement, Usability
• Removed labels: Priority-Medium, Type-Defect

[GoogleCodeExporter]

As for your first argument, why are you using strings everywhere when you can 
use the
terminals?

As for the integrity checking: I just want that in my IDE so I can use code
completion and the compiler will flag spelling mistakes as an error.

That said, you can add methods to the interface and define the enum like so:

enum Token implement IToken {
    SEMICOLON(";"), ...;

    private final String token;

    private Token(String token) {
        this.token = token;
    }

    // Method from IToken interface
    public String getToken () {
        return token;
    }
}

And if possible, it'd like this in *addition* to the String interface.

Original comment by adigu...@gmail.com on 1 Dec 2008 at 7:15

[GoogleCodeExporter]

There are still other problems, IToken would require a library, which current
implementation does not require (everything are only required for compile time).

Additionally, and more importantly, since we are using annotations for "compile"
time, not runtime, thus it is actually different in terms of how some 
information can
be made available or not.  For example, this is the reason why
CookCCOption.tokenClass is a string, not a Enum class.  See
http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=5089128 .

Also, your proposed change is actually semantically different.  For example,
specifying ';' in later grammar does not allow us to match SEMICOLON with ';'
automatically, since they could mean different precedence (like UMINUS).

I did thought about this matter a lot before I went to the current string 
approach. 
I am not saying I won't change it, but more experiments and evaluations of pros 
and
cons are needed.

Original comment by superdup...@gmail.com on 1 Dec 2008 at 8:20

[GoogleCodeExporter]

How about generating the interface at compile time if it doesn't exist?

Original comment by adigu...@gmail.com on 3 Dec 2008 at 9:02

[GoogleCodeExporter]

Could you be more specific?

Original comment by superdup...@gmail.com on 5 Dec 2008 at 5:16

[GoogleCodeExporter]

If the class IToken doesn't exist in the classpath at compile time, just create 
a
file "IToken.java" in the same package as the parser class.

If that's not possible (I'm not sure what you mean when you talk about "compile
time") put an example for the file in the docs and say "if you want this 
feature, add
this code to your project".

Original comment by adigu...@gmail.com on 7 Dec 2008 at 6:54