Minimal 3d rendering engine inside Pinocchio

nmansard commented 8 years ago

Currently, we are using Gepetto-viewer as the typically rendering for Pinocchio, which is based on OSG, which is far from being minimal, not even speaking about the Corba interface. In particular, OSG is heavy and not trivial to install.

We should maybe consider the possibility to add a minimal rendering inside Pinocchio, as a light alternative to OSG. Here are some features that seems desirable:

easy to install
cross platform: should work directly on linux, osx, and have a chance to work on MS.
the rendering would be started directly from the python shell (i.e. not like gepetto-viewer)
it would be nice that the rendering loop is not requiring to be launched from the main thread, so that it can run in the background.
it should be very light, not requiring a desktop computer to render an image.
the parsing of meshes is already done in Pinocchio, so it is not necessary that the 3D engine can read collada files.

I started looking at possible solutions. I post here some preliminary results.

Blender. Seems not possible to easily start Blender from within a python shell (i.e. no apt package proving blender as a python module (http://stackoverflow.com/questions/10972637/how-can-i-access-bpy-in-standard-python-console-bpy-is-the-blender-python-thin). Blender can be launched in background with a python console: $ blender --background --python-console This is not very nice. Moreover, the API is quite complex and only works for python 3
PyGame, based on PyOpenGL https://pythonprogramming.net/opengl-rotating-cube-example-pyopengl-tutorial/ This seems nice on ubuntu. Just apt install it (sudo apt-get install python-pygame). It seems a good candidate. Any idea if this works well on other platforms?

olivier-stasse commented 8 years ago

Blender is also used by Morse and its IK has a big problem: it does not accept joints which are at the same place (as for which the displacement is zero).

IHMO we should check if we have a sufficient rich level of functionalities. More specifically the library assimp considerable simplifies the meshes handling.

nmansard commented 8 years ago

I am just looking here for rendering: providing a mesh, give me the 3D image and display it in a dedicated window.

jcarpent commented 8 years ago

For sure, OSG is hard to install. We can easily simplify the process by using robotpkg. An other solution is to use an other over-layer of OpenGL as VTK (http://www.vtk.org) which is already used in Director for example. VTK is easy to install and has directly a python bindings.

But I really think that using Blender as a renderer is the best solution, through a corba or other middleware in between for simplicity.

nmansard commented 8 years ago

VTK might be a good solution. It is indeed straightforward to install on ubuntu. Did you try it with OSX? Would you mind to try this simple example (open up a white trackball to be rotated with the mouse): http://www.vtk.org/Wiki/VTK/Examples/Python/InteractorStyleTrackballActor

Blender does not seem to be a good option, for the reasons listed above: it is not trivial to include it in a python shell, it does not run easily with python 2.7.

jmirabel commented 8 years ago

Another cross platform option is http://www.panda3d.org/

I would say blender is not suited for your purpose. Its API is powerfull but not very simple and certainly not minimal.

olivier-stasse commented 8 years ago

@jmirabel panda3d does not provide binaries on 14.04 LT only for 16.04.

jmirabel commented 8 years ago

From the link in the install instructions, if you dig into the directory tree, you can find them: http://archive.panda3d.org/ubuntu/dists/trusty/main/binary-amd64/

So following the steps on 14.04 should work. That is in theory. I didn't try.

jcarpent commented 8 years ago

I understand all your points about Blender.

@jmirabel It seems that pand3d does not work with recent OS X versions.

@nmansard VTK in Python works well on last OS X.

olivier-stasse commented 8 years ago

my bad. That is the panda3d-runtime (plugin for browser) which is not available on 14.04. I could installed panda3d on my 14.04 and run the python samples.

olivier-stasse commented 8 years ago

The 3d model format seems to be limited to egg.

jmirabel commented 8 years ago

From http://www.panda3d.org/manual/index.php/Features, section 3D Pipeline:

Get models from your 3d modeller to Panda3D easily:

Powerful EGG/BAM format
EGG exporters for Maya, Blender and 3ds Max
Support for other 3d formats (collada, x, lwo, obj, dxf, wrl, flt)
Converters between different 3d formats and EGG

However, I did not see STL format. (more info on a way of loading them here: https://www.panda3d.org/forums/viewtopic.php?t=14063)

nmansard commented 8 years ago

According @jcarpent , Panda3D is not an option for OSX.

VTK seems the best options. They offer STL support, but not collada as far as I see. However, I would say that the best solution is to directly display the contain of geom model, as parsed by pinocchio, instead of re-opening a file with another parser. Problem is we are not parsing any material info, thus the model will be rendered in white. For minimal display, it does not seems to be a big problem to me. Is it?

thomasfla commented 7 years ago

I've just found a great demo of a web viewer. It uses JS and HTML5 so very portable; result is nice. http://robotwebtools.org/demos/urdf.html

Code source here https://github.com/robotwebtools

Paper here: http://robotwebtools.org/pdf/paper.pdf Russell Toris, Julius Kammerl, David Lu, Jihoon Lee, Odest Chadwicke Jenkins, Sarah Osentoski, Mitchell Wills, and Sonia Chernova. Robot Web Tools: Efficient Messaging for Cloud Robotics. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2015

jcarpent commented 7 years ago

Looks nice ... so nice to be true. And it seems to work with Collada and STL.

olivier-stasse commented 7 years ago

My own experience of webGL is that it is painfully slow. That was last year, maybe at that time the browsers were not using properly the video card.

nmansard commented 7 years ago

Great idea, I love it.

On 02/28/2017 04:10 PM, Olivier Stasse wrote:

My own experience of webGL is that it is painfully slow. That was last year, maybe at that time the browsers were not using properly the video card.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/stack-of-tasks/pinocchio/issues/180#issuecomment-283065335, or mute the thread https://github.com/notifications/unsubscribe-auth/AAb-CL_bnHZZjrvVrlMZfVlGklM32xRnks5rhDkAgaJpZM4Ix2ew.

jmirabel commented 7 years ago

I like the rendering as well.

I did not check whether the installation is easy. It could be a minimal viewer but it means Pinocchio must ship a Ros module. Here, minimal means a few ros packages, which already more than minimal and may be harmful to portability.

It could also be a non-minimal viewer. In terms of dependencies, it is equivalent to rviz. And it terms of functionalities, I have no idea. Although it is attractive to have the viewer in a web browser, I'm not sure it can bring anything more than rviz. Do you see any clear advantage over rviz ? I only see the capability of offering web services (like online live demos). Is this something interesting to you ?

olivier-stasse commented 7 years ago

ros is probably used as a middleware, just like omniorb. The problem in terms of portability is the lack of official support for ros on other OS than Ubuntu. Having the 3D display on web-browser would make sure that the viewer itself is supported on several OS.

nmansard commented 5 years ago

A python interface on top of webgl: http://www.glowscript.org/docs/VPythonDocs/index.html

jcarpent commented 5 years ago

I don't think it can read colada files for instance.

kabukunz commented 5 years ago

Hello, I was looking for animation systems and stumbled here. Now Panda3d has python bindings working. I'm on Os X and I tried a couple of weeks ago. Or you could embed something like libigl, which has python bindings and is able of 3d debug display Or maybe python on top of imgui like https://github.com/swistakm/pyimgui or https://github.com/podgorskiy/bimpy for something really light, but you have to add better display features upon it...

jcarpent commented 5 years ago

@kabukunz Thank you for this nice feedbacks on using Panda3d. The idea is to have something easy to install and to use with additional formats such as Collada.

jcarpent commented 5 years ago

I think we can consider this issue solved finally!

stack-of-tasks / pinocchio

Minimal 3d rendering engine inside Pinocchio #180