search

BrettMayson / HEMTT

Build System for Arma 3
http://hemtt.dev/
GNU General Public License v2.0
113 stars 40 forks source link

"IO error: stream did not contain valid UTF-8" even though the file is encoded as UTF-8 #654

Closed bastilimbach closed 8 months ago

bastilimbach commented 8 months ago

Tools (complete and add to the following information):

Description: This one is very strange... After calling hemtt release or hemtt build I sometimes get the following error:

ERROR Preprocessor error: Workspace Error: VfsError: Could not read path for '/addons/main/CfgPatches.cpp': IO error: stream did not contain valid UTF-8

If I now run the same command several times without changing any files, it succeeds. I can spam the build or release command for 10 seconds and after several errors sometimes it just works. Again, without changing any files.

I also double-checked the encoding. All files are UTF-8 encoded.

Steps to reproduce: I created a GitHub repository to reproduce the bug. You can check out the GitHub Action failure here: https://github.com/bastilimbach/hemtt-io-error/actions/runs/7951811810/job/21705671274#step:4:53

Or reproduce it locally:

  1. git clone https://github.com/bastilimbach/hemtt-io-error.git
  2. cd hemtt-io-error
  3. run hemtt release or hemtt build several times until it eventually succeeds.

Expected behavior: Always have the build succeed.

Screen capture of the behavior: hemtt

This is not speeded up. I didn't change any files. Simply pressed "up" and "enter" in the terminal.

jokoho48 commented 8 months ago

if you change the 2 files CfgPatches.cpp and CfgFunctions.cpp to hpp it works as expected as it should

BrettMayson commented 8 months ago

This has been fixed in b822c44, with a warning. config.cpp is really the only thing that should have .cpp, Includes should be using .hpp

What was happening, is it was rapifying your CfgFunctions.cpp and CfgPatches.cpp, then when it got to config.cpp and tried to include them, they were now binary blobs. Just by the nature of multithreading, occasionally it would do config.cpp first / fast enough to read the original data before they were rapified.

bastilimbach commented 8 months ago

Oh wow... That was so easy to fix. You cannot image how long I checked and tried to fix the encoding due to the error message :D Thank you so much guys!

@BrettMayson btw thanks for the explanation!