MS0103: open a .osim file and load its elements to the visualizer.

[idhamari]

This is one of the main tasks and it may take a longer time.

• VTK Widget must be used.
• Elements should be loaded to the correct location
• Two-way selection: navigator-visualizer
  • clicking on the visualizer make the element selected in the navigator and vice versa.
• All visualizer tools should be added with their functions.
• view Interaction should be possible e.g. zooming, moving, ..etc.

[idhamari]

Note:

I suggest doing this in two phases but feel free to use vtk directly if this is easier.

1. using the current OpenSim visualizer (it is openGL based).
   • this provides more understanding to the current gui pipeline.
   • this helps to keep back compatibility.
   • this also makes it easier to get new updates when needed.
2. using vtk.

[yasseerr]

• I will try to understand the current visualiser and see how to integrate it in Qt as long it is not based on jxbrowser wich uses JavaSwing wich i dont see i way to include it
• and for openGL i think QVtkOpenGlWidget a new widget instead of the QVtkWidget will offer back compatibility ---to explore

[yasseerr]

@idhamari after some reserch we extracted 3 points :

• The default visualizer is a seperate program by it own and it does not offer a library to include it in OpenGL Surface so we can not use it inside Qt the link for the program opensim_dependencies_install\simbody\bin\simbody-visualizer.exe
• One Solution is that we develop our own program that render the Models using Qt3D it consist of rebuilding opensim-visualizer from the ground
• the other solution like you mentioned is to use the current visualiser and transform it to work with QtWebEngine the equivalent of jxBrowser wich is already working but using the server of a running opensim-gui as a start the link to open a visualiser in a browser is http:/localhost:8002/threejs/editor/index.html but the work to do is to make it run from this project

how do you think we should procced , and here is a screenshot of OpenSimQt using the server of openSim-gui using QtWebEngine

[idhamari]

The default visualizer is a seperate program by it own and it does not offer a library to include it in OpenGL Surface so we can not use it inside Qt the link for the program opensim_dependencies_install\simbody\bin\simbody-visualizer.exe

the source code is available, so probably we can either:

• build it as a library using it source code.
• use its source code in our openGL qt widget

For now, I suggest to use the QtWebEngine solution if the above solutions take time, I will try to test it tomorrow and provide feedback. I have some students who may work on the

[idhamari]

@yasseerr You may start working on the next tasks as they are independent and they should not take time.

[yasseerr]

@idhamari totaly agree building a visualizer using Qt3D opengl will be inspired from the default visualiser by going over SimTK documentation https://stanford.edu/~dembia/simbody_taskspace/html/classSimTK_1_1Visualizer.html i will try and finish the QtWebEngin integration and the other tasks

[idhamari]

@yasseerr

totaly agree building a visualizer using Qt3D opengl will be inspired from the default visualiser by going over [SimTK documentation] (https://stanford.edu/~dembia/simbody_taskspace/html/classSimTK_1_1Visualizer.html)

probably we can build a shared visualizer library by modifying the simbody visualizer CMakeLists.txt , you can also ask about this in the simbody forum, check first if there are related questions.

It seems QtCreator can not recognize QWebEngineView widget, I get this error:

mainwindow.cpp:-1: error: undefined reference to `QWebEngineView::QWebEngineView(QWidget*)'

Maybe this is an error related to Qt version, what version of are you using. Mine is Qt5.9.5. I could be able to run the Qt web examples i.e. webengine and webview but they used .pro not CMake, they also uses QML. So Maybe the problem is related to CMake or QML.

[yasseerr]

@idhamari

• Sorry i didnt mentioned that , but you need to open Qt Maintenace tool and install the WebEngine Module otherwise it will not work

[idhamari]

WebEngine was already installed. I found that I forgot to update the CMakeLists.txt file so I used the old file which does not contain the WebEngine:

          find_package(Qt5 COMPONENTS Widgets WebEngine WebEngineWidgets  LinguistTools REQUIRED)

It works now but it seems I have to have a server running

[yasseerr]

Nice! the server is created when you run the netbeans app (just for demonstration)

[idhamari]

Yes, it works now, thanks for the info.

[idhamari]

@yasseerr

As I understood:

• Current simebody-visualizer uses glut which can not be integrated into Qt. There is [a file]() with around 3000 lines of code.
• opensim-gui does not use simbody-visualizer, instead, it uses a new (similar) java visualizer based on java threejs 3D library. There is [one file]() with around 2000 lines of code.
• One option is to convert the java code to cpp then use the same approach in Qt e.g. using QtWebEngine and threejs. This means we still using java.
• Another option is reimplement our visualizer using a Qt supported 3D engine e.g. Qt 3D, Ogre or VTK.

Next Todos: Creating vsSimbodyVisualizer alongside the current simebody-visualizer.

• Study the current implementation of simbody-visualizer, how it communicates with the simulation process, how it is integrated in opensim-core, how it is integrated in opensim-gui.
• The simbody source code shouldn't be modified, we just add a new folder with our visualiser. If needed you can create inherited classes from the original ones.
• vsSimbodyVisualizer is based on VTK, this makes it easier to visualize different mesh format and write less code. VTK has also large support in opensource community and is used in similar Qt-based applications e.g. 3D SLicer.
• vsSimbodyVisualizer works as a standalone application e.g. like the current visualizer.
• vsSimbodyVisualizer works as a library and can be integrated into an external application e.g. our opensimQt interface.
• vsSimbodyVisualizer should have all the functions needed for integration to an external interface.
• vsSimbodyVisualizer should have better programming readability and simplicity e.g. less code and fewer files.
• It should work with opensim-core and opensim-gui the same way as the current visualizer..
• Compare the performance of vsSimbodyVisualizer to simebody-visualizer.

[idhamari]

@yasseerr

please use our simbody branch:

https://github.com/VisSimKoblenz/simbody/tree/vsSimbodyVisualizer

You may also change the link in opensim-core dependancy Cmake to point to this link as well.

[yasseerr]

@idhamari understood !

[idhamari]

@yasseerr

Urgent

Before things get out of control as we are getting more and more files, please organize the naming of the current files in a standard way:

• use vsWord1Word2 e.g.
  • visualizervtk -----------------------> vsVisualizerVTK
  • openglvisualizer ------------------> vsOpenGLVisualizer
  • geometryimplementationqt ----> vsGeometryImplementationQT
  • actuatorsnode --------------------> vsActuatorsNode
• use the exact class and function names in your code e.g. vsVisualizerVTK
• use the same concept when creating new files and writing new code.

This will ensure readability and solve compatibility in Linux.

[idhamari]

A useful VTK book is added.

[yasseerr]

@idhamari thanks for the suggestion! just an update : its appear we can implement our visualizer by inheriting the \Simbody\Visualizer\src\VisualizerGeometry.h and implement it functions to render using vtk wich i had a problem this past days concerning the vtp files not been called in implementMeshFileGeometry however it had been solved, hopfully the geometryimplemetationqt.h will be implemented without any errors and the model will be loaded to the screen in the next 5 days without having to plug anything in the simbody project because we have access to the State variable unlike visualizer-simbody.cpp but first the naming of the files will be changed

[idhamari]

its appear we can implement our visualizer by inheriting

This sound great

i had a problem this past days concerning the vtp files not been called

If in the future you have such a problem please let me know. I could suggest we ignore the vtp file because it is only one format and can be converted to another format when needed e.g. stl

I am looking forward to testing the next working version version.

[yasseerr]

good to know, but the problem was about getting the system to load the meshes from the bodies independent from the mesh format, its about tweaking the params and calling the right functions still next version comming soon 👍

[yasseerr]

@idhamari hi! what are those violet spheres and are they a problem/bug ?

[idhamari]

@yasseerr Not sure, which model are you using? they look like markers.

[idhamari]

Notice that some model elements are visualized by opengl e.g markers, muscles and sometimes bodies. This means the mesh is generated programmatically not input from a file.

[yasseerr]

yes, so the the spheres where for debugging and they represent ground=>bodySet.. like in the topologie view , they are disabled now! the model is arm26

[yasseerr]

just some more polishing and the loading part is over.

[idhamari]

Looks great, well done!

I will test it soon and provide a feedback.

[yasseerr]

thank you, but it is not ready for testing and if you want to test it change the path in vsMainWindow.cpp to your geometry folder OpenSim::ModelVisualizer::addDirToGeometrySearchPaths("F:/FL/3/opensim-gui/opensim-models/Geometry");

[idhamari]

Good to know!

I suggest:

• add a folder called vsWorkSpace
• modify cmake file to download the folders from here in case it is empty.
• use a relative path to vsWorkSpace

[yasseerr]

ok, that will help to not set the path at each installation

[idhamari]

I also finds a few bugs related to compatibility with Linux. Please check them and consider them in your next commit (you can download the master branch as it is equivalent to this commit.

[idhamari]

@yasseerr there is something important to openGL.

Do you think it is possible to use an older VTK version that supports an older opengl? this will give a big advantage: our interface can run on older computers in addition to virtualbox.

[yasseerr]

what does the linux error say?
and for vtk what old version of vtk that support OpenGL, because we are using QvtkOpenGLWidget that should work and we are using new principles in vtk like smartPointers , so we will test this one on virtual box if it does not work we will try and use the old one if that is possible depending on the version

[idhamari]

what does the linux error say?

It is related to the file and class names you missed. Also sometimes you are using the class name when it is not needed. You can compare your branch with the master branch and check what changed.

For now, no need to worry about this. Please continue using the current version.
Unfortunately, VirtualBox supports only older versions of OpenGL.

[yasseerr]

we will work on it later but maybe you are right they do not use vtk because of backword compatibility that glu32 provide , ifound this on visualizer.h for the error try delete the build folder because a changed all the class names to vsNameName format

[idhamari]

Here is an example of the error:

in vsMainwindow.cpp you used vsMainWindow::vsMainWindow(QWidget *parent)

so changing the filename vsMainwindow.cpp to vsMainWindow.cpp fix the problem

There are some other similar problems so please use the fixed code from the master in your branch.

[idhamari]

We will continue using the new openGL for now as the hardware is getting better and the one we use is from last year. I also think virtualbox may updated their tool as well to support new openGL

[yasseerr]

hopefully.

[idhamari]

@yasseerr

I just tested it and it looks very good.

I had a problem in my system related to OpenGL but i fixed it. It appears when secure boot is enabled it disable unsigned nvidia driver. This can be solved by disabling secure boot or signing nvidia driver.

For the big joints point, you can change their size in the program so they appear smaller than the others. In the future, we can give them a different shape so they can be recognized easily.

There is still something missing: the two-way selection e.g. when you select something in the visualizer window, it should be selected in the navigator and vice versa. This is not part of the simbody-visualizer so you can find the implementation in the java interface.

[yasseerr]

good to know. for the big spheres. are you using the latest commit from visualizer-features branch because they are dealt with and muscles have been added. and for the features you mentioned it will be implemented as soon as we finish adding wrapObjects and frames ... to the scene

[idhamari]

for the big spheres. are you using the latest commit from visualizer-features branch

I will wait until you use and test the updating in the master branch as it fixe a few linux bugs.

and for the features you mentioned it will be implemented as soon as we finish adding wrapObjects and frames ... to the scene

Ok. When everything is done we will have a skype meeting to discuss the progress and close this task.

[yasseerr]

@idhamari Hello! i think all the component are loaded in the screen , and the visualizer branch has been merged with the master ,you can check the execution on linux i just noticed that you added a folder for vsWorkSpace to store the geometries but the current solution will clone it directly to the executable_folder/vsWorkSpace

[idhamari]

@yasseerr excellent.

and the visualizer branch has been merged with the master ,you can check the execution on linux

Well done! there are still two errors:

 vsVisualizerVTK.h:24:31: error: extra qualification ‘vsVisualizerVTK::’ on member ‘renderGeometry’ [-fpermissive]      
      vtkSmartPointer<vtkActor> vsVisualizerVTK::renderGeometry(OpenSim::Geometry *geometry);

This can be fixed by using:

      vtkSmartPointer<vtkActor> renderGeometry(OpenSim::Geometry *geometry);

Please consider the same practice in future code.

 Gui/Modeling/vsContactForcesNode.cpp:2:10: fatal error: vsOneContactForceNode.h: No such file or directory  #include "vsOneContactForceNode.h"
      ^~~~~~~~~~~~~~~~~~~~~~~~~

This can be fixed by changing two filenames(notice the first letter in "Force"):

  vsOneContactforceNode.h and vsOneContactforceNode.cpp to 
  vsOneContactForceNode.h and vsOneContactForceNode.cpp

[idhamari]

• please fix the above errors.
• modify the folders.
  • remove draft folder
  • OpenSimWidgets to vsWidgets, Modeling to vsModeling and Visualizer to vsVisualizer

Then merge with the master branch and close this issue.

[yasseerr]

errors fixed at commit https://github.com/VisSimKoblenz/opensimQt/commit/ffbd6ac4692427ea9f2a11343bbd01147f2dd706