Inconsistent output when generating parser code (Type vs. List<Type>)

[EBatTiVo]

Using IDEA versions 13.1.5, 13.1.6, 14. JDK: 1.6, 1.7, and 1.8 (Using default options; aggressive options appear to make it worse). Grammar-Kit versions: 1.2.0, 1.2.0.1 (Did not occur on 1.1.x versions.) OS: Windows 8.1, Linux (Ubuntu 14.04, CentOS 6.5)

The Intellij-haxe project is seeing an inconsistency in output when generating code from within the Intellij IDE (Ctrl+Shift+G). Sometimes the (consistent with v1.1.x) output looks like: HaxeFunctionType getFunctionType(); and sometimes: List<HaxeFunctionType> getFunctionTypeList(); We say "sometimes" because the bug exhibits itself intermittently: at times you can regenerate for hours and not see the bug, and other times, you will hit it the first time you generate. (The recovery mechanism is to exit and restart IDEA and /usually/ you get the expected result.)

We've been trying to automate generation, so that we don't have to check in the gen/* files, but command-line generations always give the second (unexpected) result. If you want to try using our scripts, you can find them in the tivo repository.

The relevant BNF is here. Since the bug is intermittent, it will be very difficult to create a minimal example.

The output diff is:

diff --git a/gen/com/intellij/plugins/haxe/lang/psi/HaxeTypeTag.java b/gen/com/intellij/plugins/haxe
index 8309947..3bf6410 100644
--- a/gen/com/intellij/plugins/haxe/lang/psi/HaxeTypeTag.java
+++ b/gen/com/intellij/plugins/haxe/lang/psi/HaxeTypeTag.java
@@ -25,10 +25,10 @@ import com.intellij.psi.PsiElement;

 public interface HaxeTypeTag extends HaxePsiCompositeElement {

-  @Nullable
-  HaxeFunctionType getFunctionType();
+  @NotNull
+  List<HaxeFunctionType> getFunctionTypeList();

-  @Nullable
-  HaxeTypeOrAnonymous getTypeOrAnonymous();
+  @NotNull
+  List<HaxeTypeOrAnonymous> getTypeOrAnonymousList();

 }
diff --git a/gen/com/intellij/plugins/haxe/lang/psi/impl/HaxeTypeTagImpl.java b/gen/com/intellij/plu
index 8e20f0e..4580244 100644
--- a/gen/com/intellij/plugins/haxe/lang/psi/impl/HaxeTypeTagImpl.java
+++ b/gen/com/intellij/plugins/haxe/lang/psi/impl/HaxeTypeTagImpl.java
@@ -40,15 +40,15 @@ public class HaxeTypeTagImpl extends HaxePsiCompositeElementImpl implements Haxe
   }

   @Override
-  @Nullable
-  public HaxeFunctionType getFunctionType() {
-    return findChildByClass(HaxeFunctionType.class);
+  @NotNull
+  public List<HaxeFunctionType> getFunctionTypeList() {
+    return PsiTreeUtil.getChildrenOfTypeAsList(this, HaxeFunctionType.class);
   }

   @Override
-  @Nullable
-  public HaxeTypeOrAnonymous getTypeOrAnonymous() {
-    return findChildByClass(HaxeTypeOrAnonymous.class);
+  @NotNull
+  public List<HaxeTypeOrAnonymous> getTypeOrAnonymousList() {
+    return PsiTreeUtil.getChildrenOfTypeAsList(this, HaxeTypeOrAnonymous.class);
   }

 }

[gregsh]

Please try 1.2.1. I always get (which seems to be the correct variant according to BNF):

public interface HaxeTypeTag extends HaxePsiCompositeElement {

  @NotNull
  List<HaxeFunctionType> getFunctionTypeList();

  @Nullable
  HaxeTypeOrAnonymous getTypeOrAnonymous();

}

[EBatTiVo]

LOL. So half-and-half, eh? Did you ever duplicate the error in the first place? I'll go try and report back.

[EBatTiVo]

Well, for the moment I'm getting consistent results, but not your results. Can you tell me what the git hash is for the intellij-haxe code you're testing against?

[EBatTiVo]

OK. Now I'm seeing your results, BUT we're back to inconsistent.

[gregsh]

Got it, thanks.

[EBatTiVo]

The good news is that there are only 29 commits since it last worked. The bad news is that they don't all compile on the same versions of IDEA, so iterating (a la git bisect) is painful. On top of that, it's intermittent, so testing is hit-and-miss.

Is there anything underlying the Grammar-Kit plugin (some other plugin, maybe?) that would affect its state, or a configuration that may be causing the issue?

BTW, we're using IDEA Ultimate, if ultimate vs. community might have something to do with it.

[gregsh]

No, there's no underlying plugin. Just IntelliJ and JVM. And no difference between Ultimate and Community.

[EBatTiVo]

How about interactions with the "Flash/Flex support (1.0)" plugin, or the flex libraries included from the ${idea}/plugins/flex/lib directories?

[gregsh]

You mean "JetBrains JFlex" library created for holding JFlex.jar, a lexer generator?

It has nothing to do with Adobe Flash/Flex... no flash-games, no ads, no flics while coding!

BTW How exactly do you manage to observe the inconsistency?

[EBatTiVo]

You mean "JetBrains JFlex" library created for holding JFlex.jar, a lexer generator?

Not sure, exactly. I mean that we include every jar from the /plugins/flex/lib/ directory in our plugin's SDK classpath. There isn't a JFlex.jar file in that directory. So, whatever is shipped along with IDEA, which I expect would be the compiled JFlex libs, but I'm not sure.

flex-compiler.jar                               flex-shared.jar               idea-fdb-4.5.0.20967-fix.jar
flex-jps-plugin.jar                             FlexSupport.jar               idea-flex-compiler-fix.jar
flexmojos-flex-configs-generator-server-31.jar  idea-fcsh-fix.jar             resources_en.jar
flexmojos-flex-configs-generator-server.jar     idea-fdb-3-fix.jar
flexmojos-idea-configurator.jar                 idea-fdb-4.0.0.14159-fix.jar

BTW How exactly do you manage to observe the inconsistency?

• Restart IDEA, which loads the project.
• Load the grammar/haxe.bnf file.
• Set focus to the editor window and press Ctrl+Shift+G to regenerate the parser to the project gen/ directory.
• Set focus to the project pane and select the gen->com.intellij.plugins.haxe directory
• Right click to get the context menu and select "Update Copyright..." Select "Directory ..." and OK.
• Change focus to a shell prompt (outside of IDEA) and run "git status". That should give only the files which have changed.
• In the same shell, run 'git diff' to see the changes.

Repeat this scenario a number of times, with and without restarting IDEA between regenerations until the output changes. The interesting thing about 1.2.1 vs. 1.2.0, is that the unexpected output seemed to be generated at first, and the correct (according to your analysis) output is coming after a while. That's backward from what I saw on 1.2.0, which usually gave correct (or, at least, expected) output first and moved to incorrect output after some random number of retries. I still haven't found a consistent pattern to reproduce it.