mirage / ocaml-crunch

Convert a filesystem into a static OCaml module
Other
73 stars 21 forks source link

Open files in binary mode so buffers don't underread on Windows. #54

Closed jonahbeckford closed 2 years ago

jonahbeckford commented 2 years ago

Problem

I ran into the following error on Windows:

# (cd _build/default/package/console/setup && "C:\Program Files\Git\usr\bin\env.exe" OCAMLRUNPARAM=b ocaml-crunch -m plain -o seven_z.ml assets-to-crunch)
# Generating seven_z.ml
# ocaml-crunch: internal error, uncaught exception:
#               End_of_file
#               Raised at Stdlib__buffer.add_channel in file "buffer.ml", line 273, characters 18-35
#               Called from Crunch.scan_file in file "src/crunch.ml", line 87, characters 2-33
#               Called from Crunch.walk_directory_tree.walk_dir.repeat in file "src/crunch.ml", line 49, characters 30-54
#               Called from Crunch.walk_directory_tree.walk_dir in file "src/crunch.ml", line 53, characters 17-25
#               Called from Stdlib__list.fold_left in file "list.ml", line 121, characters 24-34
#               Called from Main.walker in file "src/main.ml", line 50, characters 4-113
#               Called from Cmdliner_term.app.(fun) in file "cmdliner_term.ml", line 25, characters 19-24
#               Called from Cmdliner.Term.run in file "cmdliner.ml", line 117, characters 32-39

after using crunch with a very basic invocation

Cause

Buffer.add_channel will raise End_of_file if it does not read as much as was allocated. See https://github.com/ocaml/ocaml/blob/46c947827ec2f6d6da7fe5e195ae5dda1d2ad0c5/stdlib/buffer.ml#L271-L273

On Windows open_in will translate carriage returns and perhaps other characters.

Solution

Switch to open_in_bin. With the changes in this PR my tests worked

samoht commented 2 years ago

Thanks!