search

WeiDUorg / weidu

WeiDU is a program used to develop, distribute and install modifications for games based on the Infinity Engine.
http://www.weidu.org
GNU General Public License v2.0
90 stars 20 forks source link

Embedded WeiDU scripts provoke parser error when built with Windows-style line breaks #115

Closed Argent77 closed 6 years ago

Argent77 commented 6 years ago

If script files in src/tph/*/ contain Windows-style line breaks during the build process, the resulting WeiDU executable will fail to install mods with the following error:

[.../WEIDU_NAMESPACE/fl_functions.tpa] PARSE ERROR at line 1 column 0-1
Near Text: <<
    GLR parse error
ERROR: parsing [.../WEIDU_NAMESPACE/fl_functions.tpa]: Parsing.Parse_error
Stopping installation because of error.

ERROR Installing [Test Mod], rolling back to previous state
Unable to Unlink [A7Test/backup/0/OTHER.0]: Unix.Unix_error(1, "unlink", "A7Test/backup/0/OTHER.0")
[A7Test/backup/0/UNSETSTR.0] SET_STRING uninstall info not found
Will uninstall   0 files for [A7Test/SETUP-A7Test.TP2] component 0.
Uninstalled      0 files for [A7Test/SETUP-A7Test.TP2] component 0.
Unable to Unlink [A7Test/backup/0/READLN.0]: Unix.Unix_error(20, "unlink", "A7Test/backup/0/READLN.0")
Unable to Unlink [A7Test/backup/0/READLN.0.TEXT]: Unix.Unix_error(20, "unlink", "A7Test/backup/0/READLN.0.TEXT")
ERROR: Parsing.Parse_error

Apparently the build process adds a second CR before LF (on Windows), which WeiDU's parser doesn't like.

This issue makes working with WeiDU sources on Windows difficult, since git automatically converts line breaks to CRLF by default.

FredrikLindgren commented 6 years ago

I don't think this is WeiDU. After manually converting fl_functions.tpa to CRLF, I can't reproduce this on either of my Windows build systems (OCaml 4.01 and an older toolchain and 4.05 and a bit more recent toolchain).

Argent77 commented 6 years ago

I could track the issue to scripts/make_tph.ml, which generates src/tph.ml with the invalid \r\r\n line breaks.

With my meager OCaml knowledge I have managed to remove all \r instances from the buffer in the load_file function. The resulting src/tph.ml still contained single \r instances which suggests that a subsequent operation (maybe Printf.fprintf from main() ) converts \n to \r\n on Windows.

Btw, I'm using OCaml 4.02.3 (32-bit) from there: http://protz.github.io/ocaml-installer/

FredrikLindgren commented 6 years ago

Not that it matters much, but I would guess it's the channel opened by open_out that does a naive native conversion on line endings. OCaml policy, as such, is that line-ends are converted into LF on read and into native on write*. I would guess it's your OCaml distribution that's doing it.

*Except when it isn't. The Unix module does, or, at least, did, not, and this is what load_file uses to read.

For reference, my 4.05 is from here: https://fdopen.github.io/opam-repository-mingw/installation/ and my 4.01 is from a distribution that was discontinued before the one you are using was discontinued.

Second edit: 4.02 will also be too old if I fix the deprecated functions from the String module. Those were deprecated in 4.03. But I don't think I have to do that now; they are, to my knowledge, not (yet) build-blocking.

Argent77 commented 6 years ago

Seems to be a local issue. Closing.