search

wiredsister / OpenTransplant

Organ Procurement Transplant System for "We, The People" developed using publicly available information about Organ Transplant & cutting edge medical research.
Apache License 2.0
41 stars 1 forks source link

Make reproducible OCaml builds #6

Open wiredsister opened 3 years ago

wiredsister commented 3 years ago
yanlow commented 3 years ago

OCAML supports package version pinning like so: otuser@090b6c831cd0:/OpenTransplant$ opam pin utop 2.6.0

Turns out, pin information is stored here and is tied to your local switch configuration

otuser@090b6c831cd0:/OpenTransplant$ cat ~/.opam/ocaml-system.4.05.0/.opam-switch/switch-state
opam-version: "2.0"
compiler: [
  "base-bigarray.base"
  "base-threads.base"
  "base-unix.base"
  "ocaml.4.05.0"
  "ocaml-config.1"
  "ocaml-system.4.05.0"
]
roots: [
  "calendar.2.04"
  "cohttp-lwt-unix.2.5.4"
  "dune.2.7.1"
  "fmt.0.8.9"
  "js_of_ocaml.3.7.1"
  "js_of_ocaml-ppx.3.7.1"
  "logs.0.7.0"
  "merlin.3.4.2"
  "ocaml-system.4.05.0"
  "ocp-indent.1.8.1"
  "qcheck.0.16"
  "utop.2.6.0"
  "uuidm.0.9.7"
]
installed: [
  "angstrom.0.15.0"
  "astring.0.8.5"
  "base.v0.13.2"
  "base-bigarray.base"
  "base-bytes.base"
  "base-num.base"
  "base-threads.base"
  "base-unix.base"
  "base64.3.4.0"
  "bigarray-compat.1.0.0"
  "bigstringaf.0.7.0"
  "biniou.1.2.1"
  "calendar.2.04"
  "camomile.1.0.2"
  "charInfo_width.1.1.0"
  "cmdliner.1.0.4"
  "cohttp.2.5.4"
  "cohttp-lwt.2.5.4"
  "cohttp-lwt-unix.2.5.4"
  "conduit.2.0.2"
  "conduit-lwt.2.0.2"
  "conduit-lwt-unix.2.0.2"
  "conf-m4.1"
  "conf-pkg-config.1.3"
  "cppo.1.6.7"
  "csexp.1.3.2"
  "domain-name.0.3.0"
  "dot-merlin-reader.3.4.2"
  "dune.2.7.1"
  "dune-configurator.2.7.1"
  "easy-format.1.3.2"
  "fieldslib.v0.13.0"
  "fmt.0.8.9"
  "ipaddr.5.0.1"
  "ipaddr-sexp.5.0.1"
  "js_of_ocaml.3.7.1"
  "js_of_ocaml-compiler.3.7.1"
  "js_of_ocaml-ppx.3.7.1"
  "jsonm.1.0.1"
  "lambda-term.3.1.0"
  "logs.0.7.0"
  "lwt.5.4.0"
  "lwt_log.1.1.1"
  "lwt_react.1.1.4"
  "macaddr.5.0.1"
  "magic-mime.1.1.2"
  "menhir.20201216"
  "menhirLib.20201216"
  "menhirSdk.20201216"
  "merlin.3.4.2"
  "mew.0.1.0"
  "mew_vi.0.5.0"
  "mmap.1.1.0"
  "num.0"
  "ocaml.4.05.0"
  "ocaml-compiler-libs.v0.12.3"
  "ocaml-config.1"
  "ocaml-migrate-parsetree.1.8.0"
  "ocaml-secondary-compiler.4.08.1-1"
  "ocaml-syntax-shims.1.0.0"
  "ocaml-system.4.05.0"
  "ocamlbuild.0.14.0"
  "ocamlfind.1.8.1"
  "ocamlfind-secondary.1.8.1"
  "ocp-indent.1.8.1"
  "ocplib-endian.1.1"
  "ounit2.2.2.4"
  "parsexp.v0.13.0"
  "ppx_derivers.1.2.1"
  "ppx_fields_conv.v0.13.0"
  "ppx_sexp_conv.v0.13.0"
  "ppx_tools_versioned.5.4.0"
  "ppxlib.0.13.0"
  "qcheck.0.16"
  "qcheck-core.0.16"
  "qcheck-ounit.0.16"
  "re.1.9.0"
  "react.1.2.1"
  "result.1.5"
  "seq.0.2.2"
  "sexplib.v0.13.0"
  "sexplib0.v0.13.0"
  "stdio.v0.13.0"
  "stdlib-shims.0.1.0"
  "stringext.1.6.0"
  "topkg.1.0.3"
  "trie.1.0.0"
  "uchar.0.0.2"
  "uri.4.0.0"
  "uri-sexp.4.0.0"
  "utop.2.6.0"
  "uuidm.0.9.7"
  "uutf.1.0.2"
  "yojson.1.7.0"
  "zed.3.1.0"
]
pinned: "utop.2.6.0"

I think the most straightforward way to handle this is to take a known working configuration and install specific version numbers in the Dockerfile, and do an upgrade every now and then.

wiredsister commented 3 years ago

Yeah, @yanlow that sounds great to me.

We're kind of stuck because I can't cleanly upgrade opam on my machine for whatever reason and I'm also blocked on being able to find an actual lock file in my version 2.0.3. I'll do more digging and we can get this resolved. In the meantime, this is another vector of failure and security vulnerability. Adding a blocked label.

wiredsister commented 3 years ago

https://twitter.com/mjambon/status/1343736196522905603 👈 full answer from Martin about how to do dependency management with examples. Like I said, we avoid much drama with this issue through Dockerizing, but it's still worth solving.

yawaramin commented 3 years ago

Opam supports lockfiles, with exact version numbers of all direct and transitive dependencies. Here's a tutorial: https://khady.info/opam-sandbox.html (note, in latest opam versions opam lock is a built-in command, no longer a plugin).

The setup-ocaml GitHub action lets you set up a GitHub workflow with caching (fast builds): https://github.com/ocaml/setup-ocaml

One of its steps is: - run: opam install . --deps-only --with-test

When you have an opam lock file, you can change that to: - run: opam install . --locked --deps-only --with-test. That will install the locked dependency versions.