WebSocket support

[floe]

I'm opening this issue as a kind of reminder for myself, since we are planning on implementing WebSocket support in a current project. Pull request hopefully coming soon :-)

[mkalten]

OK, I already integrated the WebSocket code into the TUIO11 and TUIO20 libraries. I guess it would be great to add some browser examples to test everything. I'd suggest to implement a console TuioDump and graphical TuioDemo example, just like for the other environments. In any case this issue should go to the according TUIO repositories ...

[floe]

But the Reactivision codebase doesn't actually contain the WebSocket class, right? It's only part of the TUIO reference implementation right now - I was assuming that Reactivision could open a listening WebSocket by default and wait for incoming connections, parallel to the standard UDP transport.

[mkalten]

Yes, reacTIVision is still cooking its own TUIO messages without using the actual library. But I will start with the integration of the TUIO11 or TUIO20 libraries now, although I am not decided yet which I should use. TUIO11 would allow testing within many existing environments, but TUIO20 would allow the integration and implementation of many new features, and the stabilization of the library in an actual test case .... but maybe I should solve the dilemma by integrating TUIO11 backward compatibility into the TUIO20 code, or just create a simple conversion tool for that ... mmmh

[floe]

Ah, now I see. In this case, I would also suggest to switch to TUIO 2 for the next rTV release, and perhaps add a conversion tool?

[mkalten]

In both cases I'll need to reorganize the current code base substantially, so integrating TUIO11 for an intermediate 1.6 release would allow me the proper testing of everything in a stable environment, while directly jumping to TUIO20 would make that difficult for the time being. The intermediate reorganization will allow a rather quick switch to the TUIO20 code for a following 2.0 release, which may need a longer development phase anyway, since it requires the implementation of several new tracking features.

The reorganization mostly applies to the acquisition layer (portvideo) as well as the transport layer (TUIO) and some other improvements such as a better calibration and the integration of filters.

The new tracking features will concentrate on the possibility of using various fiducial types such as the current amoebas and my new yamarashi symbols as well as the integration of other types such as QR codes or an implementation of the MS byte tags for example. This kind of feature will definitely require TUIO20 already, so a switch will be necessary at this point. Also the additional contour tracking of untagged objects will not only require TUIO20 but also involve the implementation of a better threshold algorithm, which may result to be a tricky task.

From this perspective the intermediate 1.6 release based on TUIO11 still makes sense, since all those intermediate steps are anyway necessary towards the 2.0 release, and should keep things more stable on the way.

[floe]

:+1: sounds very sensible, as the APIs of TUIO 1.1 and 2.0 are not that different.

[moebiusfrost]

I'd like to know what the current status of the development is. I have difficulties with the realization of a multitouch table web application on a OS X main computer connected via wifi and hdmi with a Sur40 with Linux (which is the touch input device and main screen) and reacTIVision because I don't know yet how to get touch events into my browser (on my OS X). I already tried npTuioClient which was promising but it was not possible for me to get proper html5 touch events for any interaction (touch, drag etc.) For instance like in D3.js. So the next reasonable step would be the usage of websockets?

[mkalten]

with the integration of the TUIO11 library, the WebSocket support is now also available from within reacTIVision. It can be configured by changing the according type tag in reacTIVision.xml:

This still needs the according implementation and testing on the web client side ... anyone?

[floe]

@nomve has most of the client-side implementation done in https://github.com/nomve/Tuio.js - EDIT: corrected link.

[mkalten]

OK that's great, I will have a look at it. Should be a good starting point for the TUIO/Web client implementation. Maybe you could give it a try too, checking out the latest reacTIVision 1.6 from master. Things are wrapping up nicely now, under Linux the most significant improvements are already working.

[nomve]

Hi,

I can confirm it's generally working, but I haven't used it too much (mostly because of #13). Maybe worth noting is that there can be only one source; I tried just adding <tuio type="web" port="3334"> to the config, and it didn't work before I deleted the udp entry.

edit: I've already used it a bit more with the Tuio2Simulator and didn't notice any issues either.

[mkalten]

Thanks for testing this Goran! Yes, at the moment it is only possible to select one TUIO transport at once, but I will add multiple transport options asap. This is now just a configuration issue, since the TuioServer already support multiple transport alternatives.

[mkalten]

OK, I added the possibility for multiple TUIO transport configurations with the last commit, let me know if my comment on #13 solved your camera configuration problem ...

[nomve]

Thanks again. I'll report any issues if I run into them, but in general the WebSocket connection works fine.

[mkalten]

OK, then I will close this issue for now ... it would be great if you could prepare a minimal demo based on your code, which includes the handling of TUIO11 cursors, objects and blobs. I'd be glad to include this into the official collection of TUIO11 reference implementations. thanks, Martin

[nomve]

it would be great if you could prepare a minimal demo based on your code, which includes the handling of TUIO11 cursors, objects and blobs.

The original Tuio.js library had a couple demos that should still work, but I need to clean them up a bit. But will do.

[Jones-S]

I tried to add this to my reacTIVision.xml: <tuio type="web" port="3334"> or also <tuio type="web" host="127.0.0.1" port="3334">, but I don't get a message via websocket.

I tried to inspect with WireShark and I see that there are some things being sent around:

But the port seems to be unreachable...

Source: 127.0.0.1 Destination: 127.0.0.1 Protocol: ICMP Length: 60 Info: Destination unreachable (Port unreachable)

I also tried other ports like 6050, but no change...

Any idea why this is happening? (I know this is quite an old repo and and old issue, nonetheless, somebody might be able to help out?)

In WireShark I see that there is still a UDP message sent out on port 3334. Should this not stop if I change the tuio config in the xml to web?

[gmenegoz]

What OS are you working on? Are you using the released 1.5.1 or did you compile from scratch the 1.6 (still under development)?

[Jones-S]

Mac OS X 13.3 I am using the release, cause I did not want to dive into how I can build 1.6. If there was an easy guide I can also build 1.6 with xcode...

[gmenegoz]

I think (but I am just a random guy using this software) that websocket support is not present in 1.5.1.

I usually work on Arch Linux and Windows 10. In both OS I tried 1.5.1 and I compiled 1.6 from source code. In 1.5.1 only UDP, in 1.6 I found websocket too.

Since I actually needed websocket in 1.5.1 too, I had to employ a node server to receive UDP traffic and send it to a websocket.

Maybe I'm missing something but this is my experience