spnda / fastgltf

A modern C++17 glTF 2.0 library focused on speed, correctness, and usability
https://fastgltf.readthedocs.io/v0.8.x/
MIT License
315 stars 48 forks source link

Unicode paths in URIs are not valid #29

Closed DragonJoker closed 1 year ago

DragonJoker commented 1 year ago

fastgltf fails to parse the glTF sample project Unicode❤♻Test, due to Unicode characters in file paths in the glTF file, like this one.

spnda commented 1 year ago

What do you mean exactly by "fails to parse" in your case? I know that the code currently doesn't work on Windows for wide strings, but I'm currently fixing that.

spnda commented 1 year ago

Turns out that the code does in fact work in Windows as expected. My issue was that the file was encoded as UTF-8 but had no BOM making MSVC interpret the file as ASCII, which in turn broke the file path string. When I have proper UTF-16 characters written in the std::filesystem::path the current code works perfectly and the following test succeeds on both Ubuntu and Windows with GCC, Clang, and MSVC.

TEST_CASE("Test unicode characters", "[gltf-loader]") {
    auto lightsLamp = sampleModels / "2.0" / std::filesystem::u8path(u8"Unicode❤♻Test") / "glTF";
    fastgltf::GltfDataBuffer jsonData;
    REQUIRE(jsonData.loadFromFile(lightsLamp / std::filesystem::u8path(u8"Unicode❤♻Test.gltf")));

    fastgltf::Parser parser;
    auto asset = parser.loadGLTF(&jsonData, lightsLamp);
    REQUIRE(asset.error() == fastgltf::Error::None);
    REQUIRE(parser.validate(asset.get()) == fastgltf::Error::None);

    REQUIRE(asset->materials[0].name == u8"Unicode❤♻Material");
}

So always make sure your files are encoded as UTF-8 and the compiler properly reads the symbols.