Add mipmaps to textures in `ModelExperimental`

[j9liu]

Currently, ModelExperimental's TextureManager doesn't handle mipmapping like GltfTextureLoader does. This was started in #10158, but there turned out to be more work involved and the PR has gone inactive. A thorough implementation would do the following:

• [x] Put the resizeImageToNextPowerOfTwo function into its own file under Source/Core/resizeImageToNextPowerOfTwo.js. That way, it can be used in both GltfTextureLoader and TextureManager (and potentially be reused in the future).
• [x] Implement mipmapping in TextureManager in a similar manner to GltfTextureLoader, with some differences:
  • [x] Whereas GltfTextureLoader handles KTX2 compression with internalFormat, TextureManager handles typedArray, which is an array of numbers that represents the colors of the pixels. We can actually take these pixels and turn them into an image, then resize them, as long as the PixelDataType is of UNSIGNED_BYTE. So let's create a new file Source/Core/getImageFromTypedArray.js containing function getImageFromTypedArray(typedArray, width, height). Then, inside,
  • [x] Create a new canvas and get the context
  • [ ] Call createImageData on the context, with the original image width and height (don't convert to power of two yet)
  • [ ] Modify the ImageData's data field directly. This only requires making a for loop that sets imageData.data[i] = typedArray[i].
  • [x] Call putImageData with the image data we just created, with coordinates (0, 0)
  • [x] Return the canvas. This gives us an HTMLCanvasElement
  • [x] GltfTextureLoader calls console.warn for KTX2 compression, and in the same location in TextureManager, we want to console.warn for typed arrays. But before that, check if textureUniform.pixelDatatype === PixelDatatype.UNSIGNED_BYTE. If so, call getImageFromTypedArray and put the resulting image into resizeImageToNextPowerOfTwo. Then create a Texture from that and proceed as normal.
• [x] Add unit tests for resizeImageToNextPowerOfTwo , getImageFromTypedArray, and TextureManager

[jjhembd]

Hi @j9liu, I started a draft PR addressing most of this issue. It follows your proposed strategy, with a few slight changes:

• getImageFromTypedArray directly uses the ImageData constructor, rather than the context's createImageData method
• The logic for resizing is rearranged compared to GltfTextureLoader, to more clearly separate WebGL1 vs WebGL2 functionality

Handling the WebGL1 limitation (power-of-two dimension requirements) adds a lot of complexity. I wanted to separate out that complexity so that it can be easily removed whenever we drop WebGL1 support. The main createTexture function now looks like this:

  const texture = context.webgl2
    ? getTextureAndMips(textureUniform, image, context)
    : getWebGL1Texture(textureUniform, image, context);

and all the dimension checks and resizing are handled inside getWebGL1Texture.

I wrote unit tests for the helper functions (resizeImageToNextPowerOfTwo and getImageFromTypedArray), but I'm still working on the tests for the main TextureManager.

I'd appreciate your feedback on 2 things:

• Does the strategy in TextureManager make sense to you?
• How can I test both WebGL1 and WebGL2 contexts? The current TextureManagerSpec doesn't do this, and I'm not familiar with how to set up a context for the tests

Thanks!

[j9liu]

Hi @jjhembd ,

I'll need a moment to leave more detailed feedback on your PR, but from a glance:

• The ImageData constructor isn't supported by Internet Explorer, but we no longer support IE ~(@ggetz @sanjeetsuhag can you confirm?)~ confirmed by @sanjeetsuhag , so I think it's okay to use here.
• Using different functions to handle WebGL1 vs. WebGL2 functionality is a great idea, glad you were thinking ahead!

For testing both WebGL1 and WebGL2 contexts, you'll want to make two different scenes in the spec. The first one can be created normally with createScene(), we use WebGL1 by default. The scene with WebGL2 should be constructed like so:

      sceneWithWebgl2 = createScene({
        contextOptions: {
          requestWebgl2: true,
        },
      });

Then, in any unit test involving WebGL2, we have to check if the context was successfully requested. Otherwise, the unit test will fail. So we should put this at the beginning of the unit test:

      if (!sceneWithWebgl2.context.webgl2) {
        return;
      }

see ModelExperimentalSpec for how to do this, here and here