Closed reynir closed 1 year ago
reviewed and implemented rename
pairing with @reynir (a4b619b). good to go (though tests would be nice)
Besides the added tests I tested as well with a version of opam-mirror that exercises allocate
, set_partial
and rename
. I tested on unix target with the default sector size (I believe it is 512), and hvt with sector sizes 512, 2048 and 4096.
The unikernel checks out the git repository https://github.com/ocaml/opam-repository/ and fetches all archives (save for some known bad links) and save them to a tar filesystem. If the size is known ahead of time (http server responds with Content-Length: len
) we call allocate
with the size and a path inside /pending/...
and then incrementally write the contents as we receive them using set_partial
. Finally, once the file has been download we either commit by renaming the file to its sha256 sum or if the checksum doesn't match with the expected we rename it to /to-delete/...
.
About 14.5k archives are downloaded. Of those, about half of them have a Content-Length
header in the http response. That is, about half exercise allocate
, set_partial
and rename
. The remaining just use set
.
/to-delete/...
. I inspected a few of the files, and often the first few thousand bytes were readable and correct while the rest of the file was garbage. I didn't see a pattern.--check --verify-sha256
all files were succesfully verified. No entries were stored in /to-delete/
./to-delete/
. The files are https://github.com/ulrikstrid/ocaml-cookie/releases/download/0.1.8/session-cookie-lwt-0.1.8.tbz and https://github.com/sanette/oplot/archive/0.7.tar.gz. On manual inspection the checksums of the downloaded files match what I get downloading them with wget
and sha256sum
and they differ from the expected values in the opam files. I suspect I would see the same if I ran hvt 512 again, and the checksums changed in between running hvt 512 and the later hvt * runs. Running again with --check --verify-sha256
on hvt 4096 all checked out.The corrupt archives in unix are worrying, but I suspect they are not related to the tar-mirage changes.
Unless any concerns are raised I intend to merge this PR later today.
there are some CI test failures... would you mind to look into them?