ocaml / opam

opam is a source-based package manager. It supports multiple simultaneous compiler installations, flexible package constraints, and a Git-friendly development workflow.
https://opam.ocaml.org
Other
1.24k stars 356 forks source link

Filters for `extra-source { ... }` #5531

Open jonahbeckford opened 1 year ago

jonahbeckford commented 1 year ago

One thing that would greatly help me is a feature to conditionally download binaries but always have a fallback to compile from source. That way I could distribute most parts of the DkML Windows installer as opam packages, with speedy installs for our Windows users (and whoever else wants to take advantage of this feature) and slower source installs for anybody else.

extra-source { ... } does most of what is needed, except it doesn't have filters.

Here is a real example today from https://github.com/diskuv/dkml-compiler/blob/aff46e43c48919cf04601a6eacc4a504f0e700fc/dkml-base-compiler.opam#L203-L222:

extra-source "dl/windows_x86-ocamlc.opt.exe" {
  src: "https://github.com/diskuv/dkml-compiler/releases/download/1.2.0-prep4/windows_x86-ocamlc.opt.exe"
  checksum: "sha256=e6f3b091200c094b3f4c667a4333276a1e806b6a4dc60d3b576067f2f60b85e1"
}
extra-source "dl/windows_x86_64-ocamlc.opt.exe" {
  src: "https://github.com/diskuv/dkml-compiler/releases/download/1.2.0-prep4/windows_x86_64-ocamlc.opt.exe"
  checksum: "sha256=bd37004e7a9c94ac79c56824e056ad122d919065e7c10c390a43b5e33cece6ab"
}
extra-source "dl/linux_x86-ocamlc.opt" {
  src: "https://github.com/diskuv/dkml-compiler/releases/download/1.2.0-prep4/linux_x86-ocamlc.opt"
  checksum: "sha256=df8eb4e4d3a4e2e01ea0a37561ec999b64edb688ebed420b30451923bd68fc6b"
}
extra-source "dl/linux_x86_64-ocamlc.opt" {
  src: "https://github.com/diskuv/dkml-compiler/releases/download/1.2.0-prep4/linux_x86_64-ocamlc.opt"
  checksum: "sha256=a5d2914c0901e9bc9ca481fd4c9ea7fd8b79becc2d12dcd8c74142e43e18349f"
}
extra-source "dl/darwin_x86_64-ocamlc.opt" {
  src: "https://github.com/diskuv/dkml-compiler/releases/download/1.2.0-prep4/darwin_x86_64-ocamlc.opt"
  checksum: "sha256=98744a85497cba6670feb0f6d1e705286b5fddfa47f56ea180517733f4c6a54b"
}

Today I use those extra-source blocks in the dkml-base-compiler package to speed up the bootstrap of the ocaml compiler on the official DkML platforms. The problem is that all of those extra-source blocks are downloaded, regardless of whether the download is needed. And the problem would be worse if I expanded the idea to other packages, or if more binary-capable platforms are supported.

The proposal is to allow for filters like [ os = "win32" & arch = "x86_64" ] in:

extra-source "dl/windows_x86_64-ocamlc.opt.exe" {
  src: "https://github.com/diskuv/dkml-compiler/releases/download/1.2.0-prep4/windows_x86_64-ocamlc.opt.exe"
  checksum: "sha256=bd37004e7a9c94ac79c56824e056ad122d919065e7c10c390a43b5e33cece6ab"
} [ os = "win32" & arch = "x86_64" ]

Thanks.

jonahbeckford commented 1 year ago

FYI: The linux binary artifacts work on any Linux distro with GLIBC >= 2.28. Although it is not a high priority, it would be lovely to have opam fields linux-clib: glibc (or linux-clib: musl) and linux-clib-glibc: 2.32 so packages can filter on useful Linux attributes.

EDIT: I think I can make a conf-linux-clib package emit opam linux-clib and linux-clib-glibc variables.