search

floooh / oryol

A small, portable and extensible C++ 3D coding framework
MIT License
2k stars 200 forks source link

HttpFS on Windows sends gzip/deflate accept header, but doesn't decompress resulting data #342

Open kavika13 opened 4 years ago

kavika13 commented 4 years ago

I found this out while trying to run the Nuklear Oryol sample app.

Repro:

  1. Fetch oryol-samples and build them with fips
  2. fips run NuklearUIAdvanced

Expected: Runs just fine locally

Actual: Launches, shows typical blueish sample window, then crashes on an assert

oryol-samples\src\NuklearUIAdvanced\NuklearUIAdvanced.cc, crashes inside NKUI::EndFontAtlas(); (NuklearUIAdvanced.cc:118). This crash happens while it is trying to bake the font. It can't find any of the key font sections it seems to be looking for via magic numbers and traversal inside the ttf's data structure, and returns an error.

I'm about 90% sure this is because in winURLLoader.cc (line 76) the "Accept-Encoding: gzip, deflate" header is sent. However this by default will just get the compressed bytestream without decompressing it, according to the docs at: https://docs.microsoft.com/en-us/windows/win32/winhttp/option-flags

The result from IO::LoadResult on "data:Roboto-Regular.ttf" says it is 77536 bytes, whereas the file on disk is 145348 bytes. When I tried to download directly from the URL at http://floooh.github.com/oryol-samples/data/nkui/Roboto-Regular.ttf my browser told me it was going to be 75.7kb, but the manually downloaded file on disk matched the file in the repo at 145348 bytes, and was a binary match when I diffed them.

I think the solution is WinHttpSetOption with WINHTTP_OPTION_DECOMPRESSION and WINHTTP_DECOMPRESSION_FLAG_ALL. According to the Microsoft docs link above, this is only supported on Windows 8.1 and above. If you want to support Windows versions earlier than that, then I think either you'd have to pull in zlib yourself, or change the HTTP accept headers to not download compressed content.

To work around this problem in the cheesiest way, if I hack winURLLoader.cc (line 75 and 76) to this, the app no longer crashes for me:

                L"Connection: keep-alive";
                //L"Connection: keep-alive\r\n"
                //L"Accept-Encoding: gzip, deflate";
kavika13 commented 4 years ago

After that writeup, I see this is something you already knew about :) See #281

I do mention a couple other options besides libcurl above (set the Windows 8.1+ option to auto decompress, just don't accept compressed streams, or import zlib and decompress yourself).