Create GUI

[hjed]

We need a full GUI for controlling the robot, including GPS data and cameras. Suggestions include using inbuilt ROS gui library and using android tablets (more portable).

[hjed]

We decided to use OpenGLs GiGi (http://sourceforge.net/projects/gigi/) gui toolkit as this only depends on openGL for the graphical interface, which we will run on a saber with a cheap screen. We designed a basic GUI interface which will be a video feed overlaid with other information displayed in a hud style over the top. The only buttons will be to chose the video feed.

[hjed]

Created new branch createGUI, for this issue.

[hjed]

Ok, we are going to try this toolkit now http://www.fltk.org/ (depends on x11) or try writing a button function in glut. @sbt-tbs @YiweiHan

[YiweiHan]

fltk version 1.1 should be installed by default on ubuntu. The compile manual and example programs is in the pdf here: http://www.fltk.org/software.php Basically compile using g++ with all these extra flags: -L/usr/local/lib -lfltk -lXext -lX11 -lm -lfltk -lXext -lX11 -lm -lfltk_forms -lfltk_gl -lfltk_images

[YiweiHan]

Try something like this for different types of buttons:

include <FL/Fl.H

include <FL/Fl_Window.H

include <FL/Fl_Box.H

include <FL/Fl_Button.H

include <FL/Fl_Check_Button.H

include <FL/Fl_Light_Button.H

include <FL/Fl_Repeat_Button.H

include <FL/Fl_Return_Button.H

include <FL/Fl_Round_Button.H

int main(int argc, char *argv) { Fl_Window window = new Fl_Window(1000,700);

Fl_Box *box = new Fl_Box(20,40,300,100,"Hello, World!");
box->box(FL_UP_BOX);
box->labelfont(FL_BOLD+FL_ITALIC);
box->labelsize(36);
box->labeltype(FL_SHADOW_LABEL);

Fl_Button *button = new Fl_Button(400, 300, 100, 100, "normal");
Fl_Light_Button *lbutton = new Fl_Light_Button(400, 400, 100, 100, "light");
Fl_Round_Button *rbutton = new Fl_Round_Button(700, 400, 100, 100, "round");

window->end();
window->show(argc, argv);
return Fl::run();

} I'm not sure if overlay is possible with this though, but I haven't read through the entire manual yet.

[sbt-tbs]

glui an extension to glut may also be a possibility - see http://glui.sourceforge.net/ http://www.cs.unc.edu/~rademach/glui/ only depends on glut and opengl, but appears to have worse documentation than than FLtk, haven't yet tried it enough to decide how easy/not it is to use.

[YiweiHan]

@hjed @StephMc Are we able to use the ARM chip to test some simple code? I was thinking we should check for incompatibilities with the libraries we're using before it's too late.

[hjed]

I think we confirmed that they defiantly had opengl support, and its linux so if we only use core libraries it should be fine. Unfortunatly I won't be able to make it tomorrow, but you could probably use one of the sabers off the existing robot, but you'll have to talk to steph because I am not sure if they have graphics setup on them atm. (If they do you can test with one of the screens from the bluesat room). Also if you write any code tomorrow for the actual gui could you make sure in goes in the newGui branch of the git repo.

[hjed]

Features:

• display position (using gps?) and we need to be able to a waypoint
• display the battery
• signal
• video feeds (priority 1)
• how many sabers/ros nodes are online
• if we use ultrasonics sensors distance

[hjed]

Using gstreamer opengl plugin. Install instructions: git clone git://anongit.freedesktop.org/gstreamer/gst-plugins-gl cd gst-plugins-gl sudo apt-get install autoconf autopoint gtk-doc-tool libglew1.6-dev libgstreamermm-0.10-dev git checkout --track origin/0.10 git fetch origin ./autogen.sh make sudo make install

[hjed]

change line 250 in gst-plugins-gl/gst-libs/gst/gl/gstglwindow_x11.c so that it reads: gst_gl_window_new (guintptr external_gl_context)

line 445 to: guintptr

482: gst_gl_window_set_external_window_id (GstGLWindow * window, guintptr id)

(These are typdefs of the same thing but it fixes the compiler errors, devs probably used a different version of gcc)

[hjed]

So it looks like what we need to do is use either glimagesink or glupload classes to import the video data into opengl.

[YiweiHan]

Just a note, we've decided to stick with the glut implementation because it should be able to create the overlay effect we want. Also I found that FLTK requires a lot of class derivations to be effective, which would probably make everything tangled up.

[sbt-tbs]

Is there any current code using glimagesink or glupload at the moment, I can't find any? or is this the next step to be tested / attempted?

[hjed]

No. We spent a long time working out the best way to do this and how to install the gstreamer-opengl plugin. But we haven't got any code for that part yet.

[sbt-tbs]

okay, just wanted to make sure I wasn't just "not seeing it".

[hjed]

To setup on saber install: freeglut-dev3

[hjed]

Example using ffmpeg https://stackoverflow.com/questions/6495523/ffmpeg-video-to-opengl-texture

[hjed]

gst and opengl example https://stackoverflow.com/questions/20788499/improve-opengl-rendering-speed

[hjed]

setup catkin with gstreamer http://docs.ros.org/hydro/api/catkin/html/howto/format1/system_library_dependencies.html

[hjed]

Gstreamer build files: http://gstreamer.freedesktop.org/src/gstreamer/gstreamer-1.4.1.tar.xz ./configure make make install http://gstreamer.freedesktop.org/src/gst-plugins-bad/gst-plugins-bad-1.4.1.tar.xz ./configure make make install

[StephMc]

gstreamer install from source http://wiki.oz9aec.net/index.php/Building_Gstreamer_1.2_from_source except find the 1.4 source instead of 1.2

[YiweiHan]

Sorry guys, I'm feeling sick so I won't be coming in today, but I will be trying to redraw the shapes on the gui to make them look fancier.

[hjed]

Install before bad plugins http://gstreamer.freedesktop.org/src/gst-plugins-base/gst-plugins-base-1.4.1.tar.xz

[hjed]

TO use gstreamer: LD_LIB

[hjed]

LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib

[hjed]

gst-plugins-bad-1.4.1/tests/examples/gl/generic/cube test works.

[sbt-tbs]

according to: http://gstreamer.freedesktop.org/data/doc/gstreamer/head/manual/html/section-data-spoof.html quote " people have also expressed the wish to grab the output in a pipeline and take care of the actual output inside their application. While either of these methods are strongly discouraged, GStreamer offers support for this. Beware! You need to know what you are doing. Since you don't have any support from a base class you need to thoroughly understand state changes and synchronization. If it doesn't work, there are a million ways to shoot yourself in the foot. It's always better to simply write a plugin and have the base class manage it. See the Plugin Writer's Guide for more information on this topic. Also see the next section, which will explain how to embed plugins statically in your application. " end quote, see original page So we need to write a gstreamer plugin it seems, unless I'm misunderstanding.

[sbt-tbs]

Revised goal, video as part of control gui is not necessary. Priority is playing video, if possible for control buttons to spawn separate window that plays video that is acceptable.

[YiweiHan]

I just hate that gstreamer is so complex.

[hjed]

Wasn't video the main purpose of the GUI? Seeing as we were going to implement controls separately... If we don't do it we might as well just create a small widget that shows the small amount of other data, rather than a fully fledged gui and use the inbuilt ros functionality for displaying video (it integrates with gstreamer).

On 20/09/2014 4:50 PM, Sean Thompson wrote:

Revised goal, video as part of control gui is not necessary. Priority is playing video, if possible for control buttons to spawn separate window that plays video that is acceptable.

— Reply to this email directly or view it on GitHub https://github.com/bluesat/owr_software/issues/28#issuecomment-56258866.

[hjed]

You are misunderstanding. The gl plugin is grabbing the output from the pipeline, we aren't grabbing it raw :) Sorry I wasn't there today, I was in Melbourne for Rovers (did say last week, but not sure if people remembered).

On 20/09/2014 3:04 PM, Sean Thompson wrote:

according to: http://gstreamer.freedesktop.org/data/doc/gstreamer/head/manual/html/section-data-spoof.html quote " people have also expressed the wish to grab the output in a pipeline and take care of the actual output inside their application. While either of these methods are strongly discouraged, GStreamer offers support for this. Beware! You need to know what you are doing. Since you don't have any support from a base class you need to thoroughly understand state changes and synchronization. If it doesn't work, there are a million ways to shoot yourself in the foot. It's always better to simply write a plugin and have the base class manage it. See the Plugin Writer's Guide for more information on this topic. Also see the next section, which will explain how to embed plugins statically in your application. " end quote, see original page So we need to write a gstreamer plugin it seems, unless I'm misunderstanding.

— Reply to this email directly or view it on GitHub https://github.com/bluesat/owr_software/issues/28#issuecomment-56257100.

[hjed]

Yeah gstreamer is annoying, but I don't think we are going to find a simple solution to video short of showing it in a separate program (with the obvious disadvantages of that)

[hjed]

All that is required to show video without a gui is vlc and a button that executes a new process. We already had that working pretty much.

[sbt-tbs]

GStreamer documentation: http://gstreamer.freedesktop.org/data/doc/gstreamer/head/manual/html/index.html http://docs.gstreamer.com/display/GstSDK/Home

[hjed]

Ok, new plan is we are going to create a widget with the information on it that hovers over the top of vlc windows that have the video in

[hjed]

Created a new branch (widgetGui) which will have the code for this implementation so we can go back to the other one if we work out how to do it latter

[hjed]

New Design

[sbt-tbs]

If finding a segmentation fault involving the line ros::init(/parameters/); then the problem is likely actually use of global variables that clash with those used by ROS see issue #48 do not use global variables with identifiers battery signal but really don't use global variables at all

[sbt-tbs]

Current task Find how to play video from gstreamer with vlc

[StephMc]

http://stackoverflow.com/questions/13154983/gstreamer-rtp-stream-to-vlc

[YiweiHan]

For future reference http://en.wikipedia.org/wiki/Decimal_degrees

[YiweiHan]

Remembered an issue that Yilser raised a few weeks ago, we might need to display multiple "targets" at once on the screen, and be able to dynamically create and delete targets, since we will have multiple sample/obstacle locations in the competition. We may need to rethink how the targets will be done in the code.

[sbt-tbs]

Suggested feature (courtesy Daniel): Mark route taken on map with colour code for angle off terrain. Near level one colour and other colour(s) for more steep terrain.

[sbt-tbs]

Move button code into class to clean up global namespace.