Weekly Tasks

[vikasTmz]

July 3^rd, 2017 (Monday)

Tasks:

• Start basic Import plugin.

[vikasTmz]

July 10^th, 2017 (Monday)

Tasks:

• Begin Qt app development and discuss tasks and features to be implemented.
• Explore SFM and other techniques in Blender to reconstruct 3d models from image(s).

[s-trinh]

• Begin Qt app development and discuss tasks and features to be implemented.

The features that should be implemented for me:

• load a .cao model file in the Qt app and display the primitives
• allow the user to move / navigate / zoom around the object
• allow the user to create / edit an XML tracker config file
• allow the user to create / edit an init file: the user should be able to select directly the points he wants to used for the init file in the Qt app display
• allow the user to display the face normals, switch the normal if necessary and save the new .cao model file
• allow the user to select a primitive and edit its properties: name of the face, LOD parameters
• allow the user to load a hierarchical model file
• allow the user to create a hierarchical model file?
• allow the user to display the primitives in wireframe mode?

/cc @fspindle Feel free to correct me or add something I could have missed.

[vikasTmz]

Hello @s-trinh, just wanted to update my progress:

I've started working on the Qt app with the above mentioned features but I have a feeling this could take some time. There aren't many tutorials that covers Qt3D but there are few example projects and I've tried understanding most of the classes, modules and methods from there.

This experimental Scene Editor (qt version > 5.9) developed by Qt and another project that was used as a GUI for SFM (developed for older versions) are what I've been using for reference and aid to what is possible. Qt3D is still under development and thus tutorials are yet to be made covering all the functionalities.

What I'm trying to convey is that building this from scratch is proving to be a challenge. So is building/re-developing from existing open-source projects/experiments built by qt acceptable?

[s-trinh]

Hello,

Can you pinpoint the possible issues you are (will) facing (e.g. how to render a primitive, how to catch a mouse event, how to rotate an object, ...) to understand more the problem?

Can you link to the other project you are using for reference?

What I'm trying to convey is that building this from scratch is proving to be a challenge. So is building/re-developing from existing open-source projects/experiments built by qt acceptable?

I will discuss about this later with Fabien. In my opinion, my first brief answer is:

• we can reuse some code from other open source project as long as the license allows it
• I would prefer to reuse only small piece of code / component to avoid to have some code we cannot maintain / understand later
• of course we can took the inspiration, copy the logic of other program
• qt3d-editor looks like a too heavy project to be reused as is but some part of it can be reused in some way

[s-trinh]

In my opinion, we should first evaluate if the use of the Qt 3D library is appropriate for our project or not. I did some tests with Qt 3D: Basic Shapes C++ Example:

• camera handling should be straightforward with Qt3DRender::QCamera and Qt3DExtras::QFirstPersonCameraController
• cylinder primitive rendering should be easy with Qt3DExtras::QCylinderMesh and maybe also for circle (using a torus and a small minor radius)
• handling a click on a primitive can be done with Qt3DRender::QObjectPicker

What should be tested before anything else in my opinion:

• how to draw a line or maybe using a cylinder with a very small radius?
• how to draw a generic face?
• is it possible to load an .obj model file (Qt3DRender::QMesh should be able to load a Wavefront OBJ but I did not manage to do it)?
• how to properly transform each primitive in the scene (Qt3DCore::QTransform)?

Once we know how to do these points, the rest should be easy. If Qt 3D cannot allow us to do this easily, probably we should considerate using Qt OpenGL directly.

[vikasTmz]

Yes, I was exploring the same points. Qt3D uses qml which I'm not familiar with so I was also looking at using OpenGL directly. I'll push a working prototype covering the above points.

[vikasTmz]

is it possible to load an .obj model file (Qt3DRender::QMesh should be able to load a Wavefront OBJ but I did not manage to do it)?

Yes this is possible. So are you suggesting to convert the .cao model file to .obj and then use Qmesh?

handling a click on a primitive can be done with Qt3DRender::QObjectPicker

I'm trying to see if there is a way to find a Vertex/edge selection. Face could be done by creating a objectpicker for each by treating every face as a separate mesh/entity.

[vikasTmz]

I've managed to parse the cao file and generate edges. I used Qt3DRender::QGeometryRenderer::Lines to specify primitive type. For faces the option is to use triangles(Qt3DRender::QGeometryRenderer::Triangles) but not every .cao models are triangulated(cubesat). So should the faces be triangulated, thus leading to extra 3d lines that aren't present in the respective .cao file?

Code: vikasTmz/visp_cao_editor

[s-trinh]

The .cao model file cannot describe a triangulated face as the model-based tracker must track only the face edges. Yes, we will have to triangulate the face (e.g. Delaunay triangulation). Maybe it exists already something to do that in Qt. If we render the texture/material, these extra lines should not be visible:

[vikasTmz]

These meshes were rendered via Qmesh. But I've used Qt3DRender::QGeometryRenderer to specify the vertexdatabuffer and the primitve to be drawn. I can't seem to get Qt3DRender::QGeometryRenderer::Triangles to work as in the doc it says it needs a list of triangles, but not how this list should be constructed.

[vikasTmz]

Updates?

[s-trinh]

You can find here an example about to use Qt3DRender::QGeometryRenderer::Triangles:

    // Custom Mesh (TetraHedron)
    Qt3DRender::QGeometryRenderer *customMeshRenderer = new Qt3DRender::QGeometryRenderer;
    Qt3DRender::QGeometry *customGeometry = new Qt3DRender::QGeometry(customMeshRenderer);

    Qt3DRender::QBuffer *vertexDataBuffer = new Qt3DRender::QBuffer(Qt3DRender::QBuffer::VertexBuffer, customGeometry);
    Qt3DRender::QBuffer *indexDataBuffer = new Qt3DRender::QBuffer(Qt3DRender::QBuffer::IndexBuffer, customGeometry);

    // vec3 for position
    // vec3 for colors
    // vec3 for normals

    /*          2
               /|\
              / | \
             / /3\ \
             0/___\ 1
    */

    // 4 distinct vertices
    QByteArray vertexBufferData;
    vertexBufferData.resize(4 * (3 + 3 + 3) * sizeof(float));

    // Vertices
    QVector3D v0(-1.0f, 0.0f, -1.0f);
    QVector3D v1(1.0f, 0.0f, -1.0f);
    QVector3D v2(0.0f, 1.0f, 0.0f);
    QVector3D v3(0.0f, 0.0f, 1.0f);

    // Faces Normals
    QVector3D n023 = QVector3D::normal(v0, v2, v3);
    QVector3D n012 = QVector3D::normal(v0, v1, v2);
    QVector3D n310 = QVector3D::normal(v3, v1, v0);
    QVector3D n132 = QVector3D::normal(v1, v3, v2);

    // Vector Normals
    QVector3D n0 = QVector3D(n023 + n012 + n310).normalized();
    QVector3D n1 = QVector3D(n132 + n012 + n310).normalized();
    QVector3D n2 = QVector3D(n132 + n012 + n023).normalized();
    QVector3D n3 = QVector3D(n132 + n310 + n023).normalized();

    // Colors
    QVector3D red(1.0f, 0.0f, 0.0f);
    QVector3D green(0.0f, 1.0f, 0.0f);
    QVector3D blue(0.0f, 0.0f, 1.0f);
    QVector3D white(1.0f, 1.0f, 1.0f);

    QVector<QVector3D> vertices = QVector<QVector3D>()
            << v0 << n0 << red
            << v1 << n1 << blue
            << v2 << n2 << green
            << v3 << n3 << white;

    float *rawVertexArray = reinterpret_cast<float *>(vertexBufferData.data());
    int idx = 0;

    Q_FOREACH (const QVector3D &v, vertices) {
        rawVertexArray[idx++] = v.x();
        rawVertexArray[idx++] = v.y();
        rawVertexArray[idx++] = v.z();
    }

    // Indices (12)
    QByteArray indexBufferData;
    indexBufferData.resize(4 * 3 * sizeof(ushort));
    ushort *rawIndexArray = reinterpret_cast<ushort *>(indexBufferData.data());

    // Front
    rawIndexArray[0] = 0;
    rawIndexArray[1] = 1;
    rawIndexArray[2] = 2;
    // Bottom
    rawIndexArray[3] = 3;
    rawIndexArray[4] = 1;
    rawIndexArray[5] = 0;
    // Left
    rawIndexArray[6] = 0;
    rawIndexArray[7] = 2;
    rawIndexArray[8] = 3;
    // Right
    rawIndexArray[9] = 1;
    rawIndexArray[10] = 3;
    rawIndexArray[11] = 2;

    vertexDataBuffer->setData(vertexBufferData);
    indexDataBuffer->setData(indexBufferData);

    // Attributes
    Qt3DRender::QAttribute *positionAttribute = new Qt3DRender::QAttribute();
    positionAttribute->setAttributeType(Qt3DRender::QAttribute::VertexAttribute);
    positionAttribute->setBuffer(vertexDataBuffer);
    positionAttribute->setDataType(Qt3DRender::QAttribute::Float);
    positionAttribute->setDataSize(3);
    positionAttribute->setByteOffset(0);
    positionAttribute->setByteStride(9 * sizeof(float));
    positionAttribute->setCount(4);
    positionAttribute->setName(Qt3DRender::QAttribute::defaultPositionAttributeName());

    Qt3DRender::QAttribute *normalAttribute = new Qt3DRender::QAttribute();
    normalAttribute->setAttributeType(Qt3DRender::QAttribute::VertexAttribute);
    normalAttribute->setBuffer(vertexDataBuffer);
    normalAttribute->setDataType(Qt3DRender::QAttribute::Float);
    normalAttribute->setDataSize(3);
    normalAttribute->setByteOffset(3 * sizeof(float));
    normalAttribute->setByteStride(9 * sizeof(float));
    normalAttribute->setCount(4);
    normalAttribute->setName(Qt3DRender::QAttribute::defaultNormalAttributeName());

    Qt3DRender::QAttribute *colorAttribute = new Qt3DRender::QAttribute();
    colorAttribute->setAttributeType(Qt3DRender::QAttribute::VertexAttribute);
    colorAttribute->setBuffer(vertexDataBuffer);
    colorAttribute->setDataType(Qt3DRender::QAttribute::Float);
    colorAttribute->setDataSize(3);
    colorAttribute->setByteOffset(6 * sizeof(float));
    colorAttribute->setByteStride(9 * sizeof(float));
    colorAttribute->setCount(4);
    colorAttribute->setName(Qt3DRender::QAttribute::defaultColorAttributeName());

    Qt3DRender::QAttribute *indexAttribute = new Qt3DRender::QAttribute();
    indexAttribute->setAttributeType(Qt3DRender::QAttribute::IndexAttribute);
    indexAttribute->setBuffer(indexDataBuffer);
    indexAttribute->setDataType(Qt3DRender::QAttribute::UnsignedShort);
    indexAttribute->setDataSize(1);
    indexAttribute->setByteOffset(0);
    indexAttribute->setByteStride(0);
    indexAttribute->setCount(12);

    customGeometry->addAttribute(positionAttribute);
    customGeometry->addAttribute(normalAttribute);
    customGeometry->addAttribute(colorAttribute);
    customGeometry->addAttribute(indexAttribute);

    customMeshRenderer->setInstanceCount(1);
    customMeshRenderer->setIndexOffset(0); //customMeshRenderer->setBaseVertex(0);
    customMeshRenderer->setFirstInstance(0); //customMeshRenderer->setBaseInstance(0);
    customMeshRenderer->setPrimitiveType(Qt3DRender::QGeometryRenderer::Triangles);
    customMeshRenderer->setGeometry(customGeometry);
    // 4 faces of 3 points
    customMeshRenderer->setVertexCount(12); //customMeshRenderer->setPrimitiveCount(12);

    // Material
    Qt3DRender::QMaterial *material = new Qt3DExtras::QPerVertexColorMaterial(rootEntity);

    // Transform
    Qt3DCore::QTransform *transform = new Qt3DCore::QTransform;
//    transform->setScale(8.0f);

    // Custom mesh TetraHedron
    m_customEntity = new Qt3DCore::QEntity(m_rootEntity);
    m_customEntity->addComponent(customMeshRenderer);
    m_customEntity->addComponent(material);
    m_customEntity->addComponent(transform);

    Qt3DRender::QObjectPicker *tetraHedronOP = new Qt3DRender::QObjectPicker(m_customEntity);
    m_customEntity->addComponent(tetraHedronOP);
    QObject::connect(tetraHedronOP, &Qt3DRender::QObjectPicker::clicked,
                     this, &SceneModifier::clickTetraHedron);

You should get:

I get some issue with QObjectPicker with the custom mesh. A click outside (but not too far) of the custom mesh is also detected.

To triangulate, I was thinking about using an external library to do that. Maybe Qhull or CGAL? TetGen looks interesting as it should be a light dependency.

A fallback solution is to render 3D face points using a list of lines. To detect a click inside a face:

• detect when a click occurs
• determine to which primitive(s) (overlapping) it belongs using the primitive bounding box
• and finally use a ray tracing algorithm

[vikasTmz]

@s-trinh and @fspindle, I will not be able to spend the required mentioned hours in the proposal for the following 3 days since I'm on a break with my parents before my college starts on 29^th July. From 29^th onwards I will be able to dedicate the required hours.

[s-trinh]

@vikasTmz Ok, no problem.

[vikasTmz]

I get some issue with QObjectPicker with the custom mesh. A click outside (but not too far) of the custom mesh is also detected.

I think the mouse event is triggered on the bounding box of the object. Source

[s-trinh]

After thinking, it is not worth the effort to use Qt3DRender::QGeometryRenderer::Triangles as it requires a triangulation step for more or less the same visual result than with Qt3DRender::QGeometryRenderer::Lines (+ a possible additional dependency). Next move if you agree should be to simply use Qt3DRender::QGeometryRenderer::Lines to render the primitives when possible:

• [x] 3D lines
• [x] 3D faces from lines
• [x] 3D faces from points
• [ ] 3D cylinders (use Qt3DExtras::QCylinderMesh?)
• [ ] 3D circles (use Qt3DExtras::QTorusMesh?)

Once all the primitives can be drawn, what I see are:

• use Qt3DCore::QTransform to draw 3D cylinders and 3D circles at the correct location?
• add the click events
• add the widgets to edit a primitive (LOD parameters, primitive name)
• add the widget to create / edit an XML config file
• create a Qt3DRender::QCamera using the intrinsic parameters in the XML config file, otherwise use default values
• add an option / checkbox to display face normals
• how to choose the 3D points for the init config file?

[vikasTmz]

July 31^th, 2017 (Monday)

Tasks:

• [x] Use Qt3DRender::QGeometryRenderer::Lines to render the primitives.
• [ ] Render 3D cylinders and circles and apply respective transformation.
• [ ] Add click/hover events.
• [x] Add the widget to create / edit an XML config file

[s-trinh]

@vikasTmz This should be the last week of development. Remaining tasks are:

• [ ] Save Qt3DRender::QCamera Settings into XML config file.
• [ ] Parse correctly generic CAO model (see here).
• [ ] Display face normals.
• [ ] Switch model points when the displayed face normal is not correct.
• [ ] Fix bug: "run the application --> import CAO + save dialog make the application crash"
• [ ] Choose the 3D points for the init config file (can be done at the end when everything is ok).

[vikasTmz]

and also README for the Qt-app.