Fix various issues surfaced by fuzzer

[florczyk-fb]

This collects some fixes for issues found while fuzzing KTX parsing code through ktxTexture_CreateFromStream

• Integer truncation in memstream.c that could lead to out-of-bounds access
• Memory leaks in texture2.c caused by early function exits without freeing memory
• Out-of-bound read/write caused by using data read from the file without checking if the buffer has enough data
• valueLen becoming negative in hashlist.c if keyLen==keyAndValueByteSize

[CLAassistant]

All committers have signed the CLA.

[MarkCallow]

Thank you very much for these fixes. Please add a GitHub Action to run the fuzzer. Then we can have continuous checks.

[florczyk-fb]

Thank you very much for these fixes. Please add a GitHub Action to run the fuzzer. Then we can have continuous checks.

I'm not familiar with GitHub Actions, however this libfuzzer based harness could be used for setting up the fuzzer.

#include <ktx.h>

extern "C" int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) {
  ktxTexture* texture = nullptr;
  ktxTexture_CreateFromMemory(data, size, KTX_TEXTURE_CREATE_LOAD_IMAGE_DATA_BIT, &texture);
  if (texture != nullptr) {
    ktxTexture_Destroy(texture);
  }
}

[MarkCallow]

I've merged the test updates. Please update the CTS reference within this PR to the new HEAD commit in KTX-Software-CTS.

[MarkCallow]

Thank you for your hard work on this.

[MarkCallow]

I'm not familiar with GitHub Actions, however this libfuzzer based harness could be used for setting up the fuzzer.

#include <ktx.h>

extern "C" int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) {
  ktxTexture* texture = nullptr;
  ktxTexture_CreateFromMemory(data, size, KTX_TEXTURE_CREATE_LOAD_IMAGE_DATA_BIT, &texture);
  if (texture != nullptr) {
    ktxTexture_Destroy(texture);
  }
}

Where is the main() that runs this?

How does the fuzzer determine the failures? This function is not checking the return code from ktxTexture_CreateFromMemory. Where is the information logged?

For an Action we need a script to download the dependencies, presumably LLVMFuzzer, and a command that runs the fuzzer, displays the output and has a non-zero return code if there are issues.

[florczyk-fb]

Where is the main() that runs this?

The snippet is just a harness for testing KTX library, main() and the rest of fuzzer is implemented by LibFuzzer. Harness would have to be compiled by clang with -fsanitize=fuzzer,address options to generate fuzzer binary that could later be run.

How does the fuzzer determine the failures? This function is not checking the return code from ktxTexture_CreateFromMemory. Where is the information logged?

Fuzzer is looking for crashes when running the binary, therefore we don't need to check for the result of ktxTexture_CreateFromMemory as we are only interested in crashes, if the function returns error that's still a valid program behavior from fuzzer's perspective. Additionally the binary is instrumented with AddressSanitizer to make memory safety issues easier to trigger.

https://llvm.org/docs/LibFuzzer.html