SMT is incompatible with the Processing 2.1.1+

[segabor]

My simple TUIO test app keeps crashing since I upgraded Processing to 2.1.1. Platform: OSX 10.9.1, HW: MacBookPro

Complete stack trace:

java.lang.RuntimeException: java.lang.NullPointerException at com.jogamp.common.util.awt.AWTEDTExecutor.invoke(AWTEDTExecutor.java:58) at jogamp.opengl.awt.AWTThreadingPlugin.invokeOnOpenGLThread(AWTThreadingPlugin.java:103) at jogamp.opengl.ThreadingImpl.invokeOnOpenGLThread(ThreadingImpl.java:206) at javax.media.opengl.Threading.invokeOnOpenGLThread(Threading.java:172) at javax.media.opengl.Threading.invoke(Threading.java:191) at javax.media.opengl.awt.GLCanvas.display(GLCanvas.java:541) at processing.opengl.PJOGL.requestDraw(PJOGL.java:688) at processing.opengl.PGraphicsOpenGL.requestDraw(PGraphicsOpenGL.java:1623) at processing.core.PApplet.run(PApplet.java:2177) at java.lang.Thread.run(Thread.java:744) Caused by: java.lang.NullPointerException at processing.opengl.PJOGL.getError(PJOGL.java:1700) at processing.opengl.PGraphicsOpenGL.report(PGraphicsOpenGL.java:5355) at processing.opengl.PGraphicsOpenGL.beginDraw(PGraphicsOpenGL.java:1640) at vialab.SMT.SMTTouchManager.handleTouches(SMTTouchManager.java:40) at vialab.SMT.SMT.pre(SMT.java:1066) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at processing.core.PApplet$RegisteredMethods.handle(PApplet.java:1160) at processing.core.PApplet$RegisteredMethods.handle(PApplet.java:1153) at processing.core.PApplet.handleMethods(PApplet.java:1347) at processing.core.PApplet.handleDraw(PApplet.java:2297) at processing.opengl.PJOGL$PGLListener.display(PJOGL.java:862) at jogamp.opengl.GLDrawableHelper.displayImpl(GLDrawableHelper.java:652) at jogamp.opengl.GLDrawableHelper.display(GLDrawableHelper.java:636) at javax.media.opengl.awt.GLCanvas$10.run(GLCanvas.java:1284) at jogamp.opengl.GLDrawableHelper.invokeGLImpl(GLDrawableHelper.java:1106) at jogamp.opengl.GLDrawableHelper.invokeGL(GLDrawableHelper.java:981) at javax.media.opengl.awt.GLCanvas$11.run(GLCanvas.java:1295) at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:241) at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:733) at java.awt.EventQueue.access$200(EventQueue.java:103) at java.awt.EventQueue$3.run(EventQueue.java:694) at java.awt.EventQueue$3.run(EventQueue.java:692) at java.security.AccessController.doPrivileged(Native Method) at java.security.ProtectionDomain$1.doIntersectionPrivilege(ProtectionDomain.java:76) at java.awt.EventQueue.dispatchEvent(EventQueue.java:703) at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:242) at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:161) at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:150) at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:146) at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:138) at java.awt.EventDispatchThread.run(EventDispatchThread.java:91)

And the code that crashes:

import vialab.SMT.*;
import java.util.Set;
import java.util.HashSet;
import TUIO.TuioObject;

final int RW = 640;
final int RH = 480;

void setup() {
  size(RW,RH, P3D);
  SMT.init(this, TouchSource.MULTIPLE);
}

void draw() {
  background(51);

  rectMode(CENTER);
  for (TuioObject obj : SMT.getTuioObjects()) {
    pushMatrix();
    translate(obj.getX()*RW, obj.getY()*RH);
    rotate(obj.getAngle());
    rect(0, 0, 40, 40);
    popMatrix();
  }
}

[codeanticode]

Hey guys, I'm the responsible of the OpenGL renderers in Processing 2, I will look at this issue from my end (https://github.com/processing/processing/issues/2302) in a couple of days or so. In the meantime, let me know if you need any help, I did some refactoring in the PGL classes in 2.1.1 in order to use fewer static members, maybe this is what broke the library.

[kiwistrongis]

Huh. Our latest official release is on processing 2.0.3. I'll take a look tomorrow :).

[kiwistrongis]

So I couldn't reproduce in linux, and i don't have a mac easily available, but i did reproduce on windows with processing 2.1.1 and my latest build of smt. This is now a good excuse to get a windows vm up and running :).

[segabor]

I would help debugging if I could build SMT from scratch. Is there a wiki page or a README that describes how can I compile the whole stuff?

[kiwistrongis]

It's just a makefile. You need make and javac on your path, and your lib folder should look like this:

lib
|-- android.jar
|-- jbox2d.jar
|-- libTUIO.jar
`-- processing -> /opt/processing/core/library/
    |-- core.jar
    |-- export.txt
    |-- gluegen-rt.jar
    |-- gluegen-rt-natives-linux-amd64.jar
    |-- gluegen-rt-natives-linux-armv6hf.jar
    |-- gluegen-rt-natives-linux-i586.jar
    |-- gluegen-rt-natives-macosx-universal.jar
    |-- gluegen-rt-natives-windows-amd64.jar
    |-- gluegen-rt-natives-windows-i586.jar
    |-- jogl-all.jar
    |-- jogl-all-natives-linux-amd64.jar
    |-- jogl-all-natives-linux-armv6hf.jar
    |-- jogl-all-natives-linux-i586.jar
    |-- jogl-all-natives-macosx-universal.jar
    |-- jogl-all-natives-windows-amd64.jar
    `-- jogl-all-natives-windows-i586.jar

If you're on windows ( and not using cygwin ), you'll have to make a small change in the makefile. You have to change the colons to semicolons on line 11. It should look like this after:

cp = -cp src;bin'lib/*;lib/processing/*

The guy before me was using eclipse, but I'm not maintaining the eclipse project right now. The project files are still there, though, if you want to try it instead of using the makefile.

[kiwistrongis]

So I've kind of fixed the problem. The main source of the problem is that smt swaps out PApplet.g when drawing into the pick buffer, and PGraphicsOpenGL.beginDraw assumes PApplet.g is the main graphics object. For some reason there's an object (PGraphicsOpenGL.pgl) that's null otherwise. I've avoided it by moving calls around so that beginDraw is called before the swap.

@codeanticode I'm pretty sure PGraphicsOpenGL.checkGLThread() is bugged when the PApplet.g is swapped out. Even though it's actually on the correct thread, the glThread field in PGL is just null. If you fix that, the checkGLThread() calls that I've moved/created in PGraphicsOpenGL won't avoid the null pointer exceptions.

[codeanticode]

@KiwiStrongis thanks! I'm going through your pull request now...

[kiwistrongis]

My pull request is for mainly an indirectly related issue. I just saw that i accidentally included some code i didn't mean to. Have a look at this instead maybe?: https://github.com/KiwiStrongis/processing/compare

SMT is working (mostly) on these commits. There's still something that's happening that i don't think is SMT's fault, but i'm not sure.

[codeanticode]

ok I see... in order to fix https://github.com/processing/processing/issues/2346 you only need the changes in Runner.java, right? The rest of changes in PGraphicsOpenGL are meant to handle the problems that come up specifically with SMT, am I correct?

[kiwistrongis]

Thats correct, sorry. If you want I can fix the pull request. The important change for that issue is line 936 in Runner.java.

[codeanticode]

Yes, it would be great if you can update the pull request so it only contains that change. I will add @benfry to the thread because this is something that affects processing-app, which he is in charge of.

[kiwistrongis]

Uh, this is kinda the wrong thread, but oh well. The new and fixed pull request is processing/processing#2359.

[kiwistrongis]

So @codeanticode, I reproduced the (a?) problem without SMT. Play around with the following orders.

This doesn't draw anything:

g = buffer;
buffer.beginDraw();
buffer.rect( 10, 10, 100, 100);
buffer.endDraw();
g = temp;

This causes an null pointer exception:

buffer.beginDraw();
g = buffer;
buffer.rect( 10, 10, 100, 100);
buffer.endDraw();
g = temp;

This doesn't draws properly:

buffer.beginDraw();
g = buffer;
buffer.rect( 10, 10, 100, 100);
g = temp;
buffer.endDraw();

This causes a thread warning:

g = buffer;
buffer.beginDraw();
buffer.rect( 10, 10, 100, 100);
g = temp;
buffer.endDraw();

BTW let me know if you want me to stop @mentioning you

[codeanticode]

Hi, in general I wouldn't recommend replacing the g object with an offscreen PGraphics buffer, because g contains the renderer of the main, onscreen surface, and its beginDraw/endDraw methods are called behind the scenes to make sure that the OpenGL context is properly initialized, etc. I will look more carefully into each one of the combinations you mentioned.

And no problem, keep me posted about your findings.

[kiwistrongis]

Yeah, I'm aware of those issues, I've read through the code of that, and it's what's been causing the majority of SMT's problems :(. I've been removing all the applet.g replacements I can. The reason why it's done is so that we can redirect top level drawing calls where we want. An example is this rect() call on line 27, which we want to be sent to the pick buffer instead of the drawing back buffer. Now I wasn't involved in this feature's creation, and I'm considering recommending that we remove it to avoid problems, but I think my bosses would want to keep it.

That said (and correct me if I'm wrong), it doesn't sound like very good software engineering to redirect all calls of a class to a single object of the same class, especially when the reference to which is unreliable. It looks to me like there needs to be a better way of referencing the primary graphics context than just parent.g, which is broken as easily as g = null.

I'm aware I'm making strong statements without knowing your code base very well. Again, feel free to correct me where I'm wrong, no hard feelings.

[kiwistrongis]

Pretty much fixed as of b0809426d69e61c59f3edac95fd0f733477dc4c5