Add support for glTF KHR_materials_pbrSpecularGlossiness (Bee.glb not working)

[HO-COOH]

Context F3D using the vtkGLTFImporter to read glTF file and glb file. However, that importer does not support all feature from the glTF standard. The KHR_materials_pbrSpecularGlossiness seems badly needed, lets implement it

Suggested solution

• Investigate the details of the standard for that extension
• Implement the support in VTK for reading and rendering the extension

Skills needed

• VTK

Notes

To Reproduce

1. Export that file
2. Drag it into the latest release of f3d.

Expected behavior Renders like 3dviewer.

System Information:

• OS: Windows 11
• GPU and GPU driver: RTX3060

F3D Information Paste the content of f3d --version:

Additional context Add any other context about the problem here.

[mwestphal]

Can you share bee.glb ?

[HO-COOH]

Sure. Bee.zip

[mwestphal]

Looks like there is an issue reading this model indeed. I'm not sure what is happening, do you see this behavior with all file saved from windows 3d viewer or only with Bee ?

[HO-COOH]

Windmill.zip

Seems like yes. Just tried several others, all failed to load. Is there any kind of error reported in f3d?

[Meakk]

Is it working if you open the file with --geometry-only?

[mwestphal]

It doesn't, it fails inside the document loader. No idea why for now.

On Thu, 10 Aug 2023, 12:22 Michael MIGLIORE, @.***> wrote:

Is it working if you open the file with --geometry-only?

— Reply to this email directly, view it on GitHub https://github.com/f3d-app/f3d/issues/943#issuecomment-1672957571, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAX4BORMVHU3UIAMKAOZB3DXUSY7XANCNFSM6AAAAAA3KK62ME . You are receiving this because you commented.Message ID: @.***>

[mwestphal]

Sorry, it took me a while to check this one. Your file and others from windows 3D viewer are using a glTF extension that is not supported by VTK:

ERROR: In vtkGLTFDocumentLoaderInternals.cxx, line 1305
vtkGLTFDocumentLoader (0x55e7bbe62820): glTF extension KHR_materials_pbrSpecularGlossiness is required in this model, but not supported by this loader. Aborting

I see two issues here:

1. VTK should support this extension as it seems an important one
2. It would be better if F3D gave a better feedback in that case except for an empty window

[HO-COOH]

Sorry, it took me a while to check this one. Your file and others from windows 3D viewer are using a glTF extension that is not supported by VTK:

ERROR: In vtkGLTFDocumentLoaderInternals.cxx, line 1305
vtkGLTFDocumentLoader (0x55e7bbe62820): glTF extension KHR_materials_pbrSpecularGlossiness is required in this model, but not supported by this loader. Aborting

I see two issues here:

1. VTK should support this extension as it seems an important one
2. It would be better if F3D gave a better feedback in that case except for an empty window

Thanks for finding it out. And yes, please add this log to the message window otherwise users would think it loads successfully (because the file name is already shown in the rendering window) and find it confusing.

[mwestphal]

Thanks for finding it out. And yes, please add this log to the message window otherwise users would think it loads successfully (because the file name is already shown in the rendering window) and find it confusing.

I agree, currently you would need a debug version to get this output, which is not ideal.

[mwestphal]

related to https://github.com/f3d-app/f3d/issues/653

[mwestphal]

renamed this one and opened another one for the debug output: https://github.com/f3d-app/f3d/issues/966

[mwestphal]

Thanks for finding it out. And yes, please add this log to the message window otherwise users would think it loads successfully (because the file name is already shown in the rendering window) and find it confusing.

The log issue was fixed

[cryptosebek]

I utilize f3d to create miniature model thumbnails (128x128 pixels). It's essential for my purpose to render the model with a degree of approximation whenever feasible, rather than experiencing a complete failure due to an error.

For anybody struggling with this issue, here's a snippet I wrote for Node.JS, which essentially just replacing "KHR_materials_pbrSpecularGlossiness" with "pbrMetallicRoughness" as a fallback:

const fs = require("fs").promises;

async function readGLB(filePath) {
  const data = await fs.readFile(filePath);
  const header = {
    magic: data.readUInt32LE(0),
    version: data.readUInt32LE(4),
    length: data.readUInt32LE(8),
  };

  if (header.magic !== 0x46546c67) {
    throw new Error("File is not a valid GLB format");
  }
  let offset = 12;
  let jsonChunkLength = data.readUInt32LE(offset);
  let jsonChunkType = data.readUInt32LE(offset + 4);
  if (jsonChunkType !== 0x4e4f534a) {
    throw new Error("First chunk is not JSON");
  }
  const jsonChunk = data.slice(offset + 8, offset + 8 + jsonChunkLength);
  const json = JSON.parse(jsonChunk.toString());
  return { json, binaryData: data, jsonChunkLength, offset };
}

async function removeMaterials(glbData) {
  const { json } = glbData;
  delete json.extensionsUsed;
  delete json.extensionsRequired;
  json.materials.forEach((material) => {
    if (
      material.extensions &&
      material.extensions.KHR_materials_pbrSpecularGlossiness
    ) {
      material.pbrMetallicRoughness = {
        baseColorFactor:
          material.extensions.KHR_materials_pbrSpecularGlossiness.diffuseFactor,
        metallicFactor:
          material.extensions.KHR_materials_pbrSpecularGlossiness
            .glossinessFactor,
        roughnessFactor:
          material.extensions.KHR_materials_pbrSpecularGlossiness
            .glossinessFactor,
        baseColorTexture:
          material.extensions.KHR_materials_pbrSpecularGlossiness
            .diffuseTexture,
      };
      delete material.extensions;
    }
  });
  return { ...glbData, json };
}

async function writeGLB(modifiedGLB, outputFilePath) {
  const { json, binaryData, jsonChunkLength, offset } = modifiedGLB;
  let newJsonChunk = Buffer.from(JSON.stringify(json));
  newJsonChunk = Buffer.concat([
    newJsonChunk,
    Buffer.alloc(jsonChunkLength - newJsonChunk.length, 0x20),
  ]);
  binaryData.fill(newJsonChunk, offset + 8, offset + 8 + jsonChunkLength);
  await fs.writeFile(outputFilePath, binaryData);
}

async function patchGlb(path) {
  try {
    const glbData = await readGLB(path);
    if (
      !glbData.json ||
      !glbData.json.extensionsUsed ||
      !glbData.json.extensionsUsed.includes(
        "KHR_materials_pbrSpecularGlossiness"
      )
    ) {
      return;
    }
    const modifiedGLB = await removeMaterials(glbData);
    await writeGLB(modifiedGLB, path);
  } catch (error) {
    console.error("Error:", error);
  }
}

module.exports = { patchGlb };