Provide Sample Showcasing Ability To Load Complete Model Information

[kklouzal]

http://guillaumeblanc.github.io/ozz-animation/documentation/geometry_runtime/ The following is stated: "Note that ozz does not propose any production ready tool to load meshes or any mesh format. See fbx2skin from ozz sample framework for an example of how to import a skinned mesh from fbx." None of the current samples or documentation explain how to retrieve vertex, index, uv, etc.. data from models. It is to my understanding after reading the provided documentation and studying the provided samples that you are required to load this information into an application using other means?

https://github.com/guillaumeblanc/ozz-animation/blob/master/samples/framework/tools/fbx2mesh.cc Appears to showcase how to retrieve this information but the file is over 1000 lines and after going through it a few times leaves me wishing for a proper sample explaining the process.

Could it be possible to provide a sample explaining how to retrieve this data from our loaded models?

[guillaumeblanc]

Hi,

fbx2skin was renamed sample_fbx2mesh some time ago, while documentation wasn't updated. It's done now, thanks!

fbx2mesh is a command line tool which serializes mesh data as ozz::sample::Mesh in a file.

If your objective is to read meshes imported from fbx with sample_fbx2mesh, then you'll need to deserialize/load them like samples are doing. Vertex and tesselation information are then directly accessible from ozz::sample::Mesh.

I think you understood that, but as a reminder this pipeline was done for samples needs only, it's not a library feature. You might want to start form there though and extend it to your needs.

Hope it helps, Guillaume

[kklouzal]

I've been going over and over the provided sample framework and also studying fbx2mesh. I'm getting confused because a few of the samples are rendering a mesh and when you look inside the media folder for these samples there is a 'mesh.ozz' file. I'm just trying to wrap my head around how this is being executed in the framework.

This is an amazing library, the documentation does a decent job of explaining how things work, and the provided sample code is extensive in showcasing features. Unfortunately for me when I look at the sample code there is so much going on I can't quite wrap my head around it.

If there is a way to extract vertex/index/etc.. data using ozz then I want to take that route. If I need to load the original .fbx file in to get this information then I will do it that way. I just want to make sure i'm approaching the problem properly and taking the correct approach.

I have skeleton and animation data loaded with animations playing without a problem, it's time to apply them to a physical model and I can't visualize a clear starting point to accomplish this.

[kylawl]

Hi! I'm confused as to what you're trying to accomplish? The sample_fbx2mesh will load an fbx file and serialize (like the mesh.ozz file you're referring to). This is a representation of a mesh which you can load using LoadMeshes() https://github.com/guillaumeblanc/ozz-animation/blob/3e86c6f27fb5714e24738da5de7b6d3776099b93/samples/framework/utils.cc#L401 and then use to initialize buffers for your favorite graphics library. Are you struggling to construct vertex and index buffers for rendering with that data? Or are you looking to fully parse and process the fbx files yourself into your own datastructures?

As far as the complexity of the sample converter. There is unfortunately no "simple" way to demonstrate traversing and building meshes from fbx files (or any other reasonably useful scene format for that matter). There's a minimum amount of code necessary to grab everything you need and turn out something useful. My suggestion would be to run fbx2mesh in a debugger and follow through to get an idea of the structure. You'll find that it's mostly boilerplate for reading data from the fbx file.

The basic idea is

• Load the fbx file in
  • Iterate all the nodes in the scene
    • If the node is a mesh, extract mesh data (this is where it gets really messy)
    • If the node is a bone, add it to a skeleton
  • Once you're done processing the mesh and the skeleton, you can build an inverse bindpose for skinning the mesh
  • Serialize the mesh and bindpose so you can load it in your game.

There are many flavours but they all taste similar to this

Good luck,

/ Kyle

On Fri, Nov 8, 2019 at 2:33 PM Kyle Klouzal notifications@github.com wrote:

I've been going over and over the provided sample framework and also studying fbx2mesh. I'm getting confused because a few of the samples are rendering a mesh and when you look inside the media folder for these samples there is a 'mesh.ozz' file. I'm just trying to wrap my head around how this is being executed in the framework.

This is an amazing library, the documentation does a decent job of explaining how things work, and the provided sample code is extensive in showcasing features. Unfortunately for me when I look at the sample code there is so much going on I can't quite wrap my head around it.

If there is a way to extract vertex/index/etc.. data using ozz then I want to take that route. If I need to load the original .fbx file in to get this information then I will do it that way. I just want to make sure i'm approaching the problem properly and taking the correct approach.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/guillaumeblanc/ozz-animation/issues/80?email_source=notifications&email_token=ABFY7V27QZEX4F7Z6KS3HP3QSW5J5A5CNFSM4JF4DC72YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEDTEGPY#issuecomment-551961407, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABFY7V3GS5XNVSAISAJ2DGDQSW5J5ANCNFSM4JF4DC7Q .

[kklouzal]

Sorry for the late reply. Here is where my confusion was:

I had assumed that an .ozz file created with fbx2ozz contained vertex data about the model which I could in-turn extract and use to render the model itself. It does not. As the documents state this is an animation library and does not directly facilitate the rendering of 3d models. That's fine, my ignorance into what this library did and did not do lead me in the wrong direction.

I have since directly used the AutoDesk FBX SDK to retrieve per-vertex data including vertex position, bone id's and bone weights. I have also successfully rendered my models like this.

Now I am stuck getting bone transformation matricies out of OZZ and into a glm::mat4[] for upload to my skinning shader.

    void updateUniformBuffer(const uint32_t &currentImage) {
        static auto startTime = std::chrono::high_resolution_clock::now();

        auto currentTime = std::chrono::high_resolution_clock::now();
        float time = std::chrono::duration<float, std::chrono::seconds::period>(currentTime - startTime).count();

        UniformBufferObject ubo = {};
        ubo.model = glm::rotate(glm::mat4(1.0f), time * glm::radians(30.0f), glm::vec3(0.0f, 0.0f, 1.0f));

        controller_.Update(animation_, 0.1f);

        // Samples optimized animation at t = animation_time_.
        ozz::animation::SamplingJob sampling_job;
        sampling_job.animation = &animation_;
        sampling_job.cache = &cache_;
        sampling_job.ratio = controller_.time_ratio();
        sampling_job.output = make_range(locals_);
        if (!sampling_job.Run()) {
            printf("Sampling Job Failed\n");
            return;
        }

        // Converts from local space to model space matrices.
        ozz::animation::LocalToModelJob ltm_job;
        ltm_job.skeleton = &skeleton_;
        ltm_job.input = make_range(locals_);
        ltm_job.output = make_range(models_);
        if (!ltm_job.Run()) {
            printf("LocalToModel Job Failed\n");
            return;
        }

        //  Somehow translate bones into our uniform buffer
        auto poses = skeleton_.joint_bind_poses();
        for (int i = 0; i < poses.count(); i++) {
            glm::mat4 BoneMat(poses[i].identity);
            ubo.bones[i] = BoneMat;
        }

        _Mesh->updateUniformBuffer(currentImage, ubo);
    }

Can anyone shed light on how to elaborate on this section:

        //  Somehow translate bones into our uniform buffer
        auto poses = skeleton_.joint_bind_poses();
        for (int i = 0; i < poses.count(); i++) {
            glm::mat4 BoneMat(poses[i].identity);
            ubo.bones[i] = BoneMat;
        }

ubo.bones is a glm::mat4[64] which should contain final matricies for each bone after computation through OZZ. https://stackoverflow.com/questions/59032693/convert-ozzmathfloat4x4-to-glmmat4

[kylawl]

I had assumed that an .ozz file created with fbx2ozz contained vertex data about the model which I could in-turn extract and use to render the model itself. It does not. As the documents state this is an animation library and does not directly facilitate the rendering of 3d models. That's fine, my ignorance into what this library did and did not do lead me in the wrong direction.

While the fbx2ozz tool doesn't. The fbx2mesh sample does exactly what your talking about.

Can anyone shed light on how to elaborate on this section

Your models_ array should contain the bone poses required to pose the mesh. Try indexing into that instead of the skeleton. glm::mat4 BoneMat(models_[i]); instead of glm::mat4 BoneMat(poses[i].identity); You'll need to write a little conversion function rather than passing it to the mat4 constructor directly. You can read each row from one into the other. Not sure the row/column order of the open gl stuff so you may need to transpose the matrix as well.

Edit: You can find the example of how to deal with extracting the pose for open gl here https://github.com/guillaumeblanc/ozz-animation/blob/3e86c6f27fb5714e24738da5de7b6d3776099b93/samples/framework/internal/renderer_impl.cc#L1135

[kklouzal]

@kylawl Thank you for your help, I really appreciate it. _transform is just passed to another function call https://github.com/guillaumeblanc/ozz-animation/blob/3e86c6f27fb5714e24738da5de7b6d3776099b93/samples/framework/internal/renderer_impl.cc#L1248

Sadly I cannot go straight from an ozz::math::float4x4 to a glm::mat4 which is to be expected. A simple example on how to extract the needed bits out of a float4x4 might get me going in the proper direction. I wish I had a better understanding behind this but it is my first time and i'm learning as I go.

As far as fbx2mesh goes, I have looked through and studied the code behind it and I just couldn't wrap my head around everything that was going on. I think it will be more advantageous of me to do it as I am now and use the FBX SDK directly to acquire what I need from my .fbx files while letting ozz handle the animation portion. I may look into this again in the future but for now I can get what I need using the FBX SDK.

Thank you again.

EDIT:

This gets some output back on the screen, so I think i'm going in the right direction at least. It's basically garbage output though.

void updateUniformBuffer(const uint32_t &currentImage) {
    static auto startTime = std::chrono::high_resolution_clock::now();
    auto currentTime = std::chrono::high_resolution_clock::now();
    float time = std::chrono::duration<float, std::chrono::seconds::period>(currentTime - startTime).count();

    UniformBufferObject ubo = {};
    ubo.model = glm::rotate(glm::mat4(1.0f), time * glm::radians(30.0f), glm::vec3(0.0f, 0.0f, 1.0f));

    controller_.Update(animation_, time* 1.0f);

    // Samples optimized animation at t = animation_time_.
    ozz::animation::SamplingJob sampling_job;
    sampling_job.animation = &animation_;
    sampling_job.cache = &cache_;
    sampling_job.ratio = controller_.time_ratio();
    sampling_job.output = make_range(locals_);
    if (!sampling_job.Run()) {
        printf("Sampling Job Failed\n");
        return;
    }

    // Converts from local space to model space matrices.
    ozz::animation::LocalToModelJob ltm_job;
    ltm_job.skeleton = &skeleton_;
    ltm_job.input = make_range(locals_);
    ltm_job.output = make_range(models_);
    if (!ltm_job.Run()) {
        printf("LocalToModel Job Failed\n");
        return;
    }

    //  Somehow translate bones into our uniform buffer
    auto joints = skeleton_.num_joints();
    printf("UBO Joints %i\n", joints);
    for (int i = 0; i < joints; i++) {
        //  num_joints is higher than model bone count for some reason..
        if (i >= bindPoses.size()) { break; }
        FbxAMatrix BoneBind = bindPoses[i];
        auto inverse = BoneBind.Inverse();
        glm::mat4 mBone;
        auto glm_row1 = &mBone[0];
        auto ozz_row1 = models_[i].cols[0];
        glm_row1->x = ozz_row1.m128_f32[0] * inverse.mData[0].mData[0];
        glm_row1->y = ozz_row1.m128_f32[1] * inverse.mData[0].mData[1];
        glm_row1->z = ozz_row1.m128_f32[2] * inverse.mData[0].mData[2];
        glm_row1->w = ozz_row1.m128_f32[3] * inverse.mData[0].mData[3];

        auto glm_row2 = &mBone[1];
        auto ozz_row2 = models_[i].cols[1];
        glm_row2->x = ozz_row2.m128_f32[0] * inverse.mData[1].mData[0];
        glm_row2->y = ozz_row2.m128_f32[1] * inverse.mData[1].mData[1];
        glm_row2->z = ozz_row2.m128_f32[2] * inverse.mData[1].mData[2];
        glm_row2->w = ozz_row2.m128_f32[3] * inverse.mData[1].mData[3];

        auto glm_row3 = &mBone[2];
        auto ozz_row3 = models_[i].cols[2];
        glm_row3->x = ozz_row3.m128_f32[0] * inverse.mData[2].mData[0];
        glm_row3->y = ozz_row3.m128_f32[1] * inverse.mData[2].mData[1];
        glm_row3->z = ozz_row3.m128_f32[2] * inverse.mData[2].mData[2];
        glm_row3->w = ozz_row3.m128_f32[3] * inverse.mData[2].mData[3];

        auto glm_row4 = &mBone[3];
        auto ozz_row4 = models_[i].cols[3];
        glm_row4->x = ozz_row4.m128_f32[0] * inverse.mData[3].mData[0];
        glm_row4->y = ozz_row4.m128_f32[1] * inverse.mData[3].mData[1];
        glm_row4->z = ozz_row4.m128_f32[2] * inverse.mData[3].mData[2];
        glm_row4->w = ozz_row4.m128_f32[3] * inverse.mData[3].mData[3];
        ubo.bones[i] = mBone;
    }

    _Mesh->updateUniformBuffer(currentImage, ubo);
}

[kylawl]

So this looks like your example here should work. However, I have no idea what the return value of mBone[]. Are we sure it's a reference? Otherwise auto glm_row1 is going to be a value on the stack and will never make it back into mBone when you assign it to ubo.bones[i]. Second, you may need to transpose the matrix using glm's matrix transpose function. Third are you where are you applying your inverse bind pose?

[kklouzal]

ubo.bones[i] = mBone copies the value of mBone into ubo.bones[i] and ubo is an object that lives long enough for it's data to be copied over to the GPU.

I was going to add the bind pose after I got some triangles moving around but I went ahead and added it now. Check the above post for updated code.

Here is my current output, only half a model laying down. For some reason soa_num_joints is a bit larger than the amount of bones extracted from the model. I'm using alain_skeleton.ozz for the skeleton, alain_walk.ozz for the animation, and arnaud.fbx for the model which are all assets included with OZZ.

EDIT: After you mentioned about the return value of mbone[] I checked with the visual studio debugger and the values were not making it into ubo.bones. After making auto glm_row1 = &mBone[0]; references the values make it in just fine. Now I have a bunch of triangles flying and dancing around ;D

[kylawl]

The horizontal rotation is likely just an up-axis issue. Your renderer is are zup and your model is Y. Not sure what the deal with your mesh is. Maybe something to do with importing a mesh with mirrored UVs? Maybe miss calculating prim counts?

For some reason soa_num_joints is a bit larger than the amount of bones extracted from the model.

Ya the soa joints count will always be rounded to the nearest 4 joints. This is part of the simd optimized evaluation in the jobs.

Your best bet now is to debug draw some lines for the skeleton so you can see how the skeleton is or isn't aligned with your mesh.

Good luck

[kklouzal]

Thank you again for all your help, I wouldn't be this far without it! I'm sure it's because I need to glm::transpose and/or my conversions over to glm::mat4 are incorrect.

I've raised an issue on stackoverflow, as complicated as it might be, hopefully someone will give a bit of insight here. https://stackoverflow.com/questions/59032693/convert-ozzmathfloat4x4-and-fbxsdkfbxdouble4-to-glmmat4-for-gpu-sk