a-scie / jump

A Self Contained Interpreted Executable Launcher
Apache License 2.0
48 stars 7 forks source link

Avoid digesting lift manifest on every scie run. #11

Closed jsirois closed 1 year ago

jsirois commented 1 year ago

Currently, the lift manifest is digested on every run of a scie even if it already has a hash recorded: https://github.com/a-scie/jump/blob/12d714431e929457be315502485daaae4e3e68d2/jump/src/lift.rs#L173-L204

Although informal timings put the cost at ~4us, it's hashing that is not needed. Much like the file entries themselves, the hash only need be checked on extraction, which, for lift manifests, is only when the scie.boot is involved (see #7).

Typical informal timings in a noop dispatch timer test rig of:

{
  "scie": {
    "lift": {
      "name": "timer",
      "boot": {
        "commands": {
          "": {
            "exe": "/usr/bin/true",
            "additional_files": [
                "node-v18.12.0-linux-x64.tar.xz"
            ]
          }
        }
      },
      "files": [
        {
          "name": "node-v18.12.0-linux-x64.tar.xz"
        }
      ]
    }
  }
}
$ time RUST_LOG=debug ./timer
[DEBUG TimerFinished] assemble(), Elapsed=2.648µs
[DEBUG TimerFinished] digest(), Elapsed=3.395µs
[DEBUG TimerFinished] load_scie(), Elapsed=60.023µs
[DEBUG TimerFinished] new(), Elapsed=4.722µs
[DEBUG TimerFinished] unpack_archive(), Elapsed=2.554µs
[DEBUG jump::installer] Cache hit /home/jsirois/.nce/9429e26d9a35cb079897f0a22622fe89ff597976259a8fcb38b7d08b154789dc/node-v18.12.0-linux-x64.tar.xz for File { name: "node-v18.12.0-linux-x64.tar.xz", key: Some("node"), size: 0, hash: "9429e26d9a35cb079897f0a22622fe89ff597976259a8fcb38b7d08b154789dc", file_type: Archive(CompressedTar(Xz)), always_extract: false }
[DEBUG TimerFinished] prepare(), Elapsed=14.846µs
[DEBUG TimerFinished] prepare_boot(), Elapsed=103.26µs

real    0m0.001s
user    0m0.001s
sys     0m0.000s
$ RUST_LOG=debug time -v ./timer
[DEBUG TimerFinished] assemble(), Elapsed=2.645µs
[DEBUG TimerFinished] digest(), Elapsed=3.436µs
[DEBUG TimerFinished] load_scie(), Elapsed=50.202µs
[DEBUG TimerFinished] new(), Elapsed=4.473µs
[DEBUG TimerFinished] unpack_archive(), Elapsed=2.45µs
[DEBUG jump::installer] Cache hit /home/jsirois/.nce/9429e26d9a35cb079897f0a22622fe89ff597976259a8fcb38b7d08b154789dc/node-v18.12.0-linux-x64.tar.xz for File { name: "node-v18.12.0-linux-x64.tar.xz", key: Some("node"), size: 0, hash: "9429e26d9a35cb079897f0a22622fe89ff597976259a8fcb38b7d08b154789dc", file_type: Archive(CompressedTar(Xz)), always_extract: false }
[DEBUG TimerFinished] prepare(), Elapsed=14.854µs
[DEBUG TimerFinished] prepare_boot(), Elapsed=93.324µs
        Command being timed: "./timer"
        User time (seconds): 0.00
        System time (seconds): 0.00
        Percent of CPU this job got: 92%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.00
        Average shared text size (kbytes): 0
        Average unshared data size (kbytes): 0
        Average stack size (kbytes): 0
        Average total size (kbytes): 0
        Maximum resident set size (kbytes): 2816
        Average resident set size (kbytes): 0
        Major (requiring I/O) page faults: 0
        Minor (reclaiming a frame) page faults: 164
        Voluntary context switches: 1
        Involuntary context switches: 0
        Swaps: 0
        File system inputs: 0
        File system outputs: 0
        Socket messages sent: 0
        Socket messages received: 0
        Signals delivered: 0
        Page size (bytes): 4096
        Exit status: 0
jsirois commented 1 year ago

Aha, right. This is ~impossible. The lift manifest doesn't current have its size and hash stored. The size could be stored but not the hash - since the hash value would change the hash!