Display

[jchailloux]

Link to a toolkit that use GNU Plot to display near real-time data

https://www.ibm.com/developerworks/community/forums/html/topic?id=6c3d3084-38f3-4177-a45e-1cbce7a017f5&ps=25

[ddebrunner]

Are you just providing info that this toolkit exists, or are proposing it be brought into IBMStreams?

[jchailloux]

Both, I wrote this toolkit for PoC and would like to share it

[jchailloux]

I started a repository : https://github.com/jchailloux

[ddebrunner]

A common approach to visualizing realtime data has been to use the HTTPTupleView operator and then one has the power of any JavaScript visualization library, such as Dojo, D3, Google Charts etc. This also means that the live data views are browser based rather than running in the display of one of the hosts running the streams application.

HTTPTupleView is now part of streamsx.inet, here are a couple of writeups based on the orignal toolkit from Streams Exchange (the concepts still apply):

https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/W4671426578cc_49bd_b585_3b9911c491b8/page/HTTPTupleViewExamples

https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/W4671426578cc_49bd_b585_3b9911c491b8/page/More%20HTTPTupleViewExamples

[jchailloux]

HTTPTupleView has some limitations when you have to display scientific data.

GNUPlot is really powerfull for that.

What about having some new parameters in the HTTPTupleView that could use GNUPLot to create an image (set terminal jpg).

We can then display this image using the standard approach of HTTPTupleView.

[ddebrunner]

What limitations for scientific data.does HTTPTupleView have?

[jchailloux]

I was not able to display data that comes from sonar or sound spectrum

here is sample of display I would like

[ddebrunner]

HTTPTupleView just makes tuples available as JSON data. How you visualize the data depends on which Javascript library you use.

Was there missing functionality in HTTPTupleView itself?

[ddebrunner]

-1 on this toolkit in its current form for these reasons:

a) Does not support distributed without a wrapper, which means it's not supporting the main execution mode for Streams. b) Having a PE use the X11 display of a client machine does not seem the correct approach.

I can see value in using gnuplot for visualization but it needs to fit the typical production deployment of Streams application, possibly including when Streams is running in a cloud environment. My feeling is though that HTTPTupleView and a suitable JavaScript visualization library is an approach that already works.

It seems to me that this is more of a client application, one that can access data from Streams and display it locally using gnuplot.

[jchailloux]

If HTTPTupleView only provides data in JSON format I would use the JSONHelper.

From my point of view, HTTPTupleView convert data into JSON in order to use with a web application server (jetty).

I would like to use the Web App capability to display graphs that was made by GNUPlot as an image.

Then I would say that the HTTPTupleView should be able to get the data, send them to gnuplot to create a image and then display the image.

Or it's an other specific Toolkit

[ddebrunner]

If HTTPTupleView only provides data in JSON format I would use the JSONHelper.

Can you explain what you mean here?

[jchailloux]

I see you points and agree on point a. On point b X11 is not the only way to display.

I will just let this toolkit live outside GitHub/IBMStreams as it's an easy way to have a complex application to display data during PoCs and Demos when needed.

I don't totally agree with your Client application point as the HTTPTupleView also looks like an Client Application such as Cognos RTM. But never mind.

I already used HTTPTupleView found it very usefull in some use case that's why I suggested to improve the chart capability.

[jchailloux]

I mean that TupleToJSON already provide the capability to have data in a JSON format. So the use of HTTPTupleView is more than that.

[ddebrunner]

So i think ideally to make reusable functionality, one could add:

A) An operator in the com.ibm.streamsx.inet.rest namespace that accepted tuples containing the contents for an image and makes them available through the jetty web-server. Basically a blob version of HTTPXMLView aimed at serving up images.

B) an operator that created an blob of a gnuplot image.

This then allows other image generators to be developed, rather than hard-coding a gnuplot image server.

[jchailloux]

Fair points

On the B point I can propose

• Having the operator with a parameter for the terminal (jpg,png,....) that create an ouput view on the output port.
• Having an other version or let the operator able to display using X11 if no output port has been defined

Then we have to think about creating an operator has you said to deal with images(blob or not).

[mikespicer]

An operator to create an image blob from stream data input using GNUPlot seems like a good idea. We could write that blob to a file or other output using existing sinks.

For HTTP serving of images do we need a new operator or can the existing tuple view one using a reference to the image file either created directly for a blob value or a composite which also has a FileSource?

[ddebrunner]

I think a new operator, there's no need to write the blob to a file, it can be maintained in memory.

The operator can actually be generic, it would just take a blob/rstring make it available, the MIME type could be a parameter (or from a tuple), thus it is general purpose and not specific to images.

The difference that HTTPTupleView has a JSON object containing a set of tuples as the contents of the reply to a GET request. This new operator makes a tuple's attribute the complete contents of the request, as-is. It's exactly what HTTPXMLView does, it makes the contents of an XML attribute available as the contents of the GET reply, as-is.

[mikespicer]

OK that makes sense.

[chanskw]

Going through some of the work items here. Do you think this should be combined into the streamsx.visualization project?

[chanskw]

Closing this one due to lack of activity. Please reopen if you still want to contribute.