Improve testing: the ci should install more packages that depend on ocamlbuild

hhugo commented 2 months ago

In particular, we could test all packages that depend on ocamlbuild in their latest version:

opam list --depends-on ocamlbuild --column name,version,all-versions -a --no-depexts
# Packages matching: available & depends-on(ocamlbuild)
# Name                             # Version     # Versions
0install                           2.12.3        2.6.2  2.8  2.9.1  2.10  2.11  2.12  2.12.1  2.12.3  2.14  2.14.1  2.15.1  2.15.2  2.16  2.17  2.18
ANSITerminal                       0.7           0.6  0.6.2  0.6.3  0.6.4  0.6.5  0.7  0.8  0.8.1  0.8.2  0.8.3  0.8.4  0.8.5
abella                             2.0.7         2.0.2  2.0.3  2.0.4  2.0.5  2.0.6  2.0.7  2.0.8
abt                                0.0.3         0.0.3
afl-persistent                     1.0           1.0  1.1  1.2  1.3  1.4
....

In the example above, we should test abt

hhugo commented 2 months ago

The following command should return all the packages we might want to install. opam list --depends-on ocamlbuild --column name,version,all-versions -a --no-depexts | head -n-2 | sed 's/ */ /'g | awk -F' ' '{print $2==$NF?$1"."$2:""}' | grep -v "^$"

See the list

``` abt.0.0.3 alberto.0.4 aliases.0.0.5 aperf.0.1.2 apron.v0.9.14 arakoon.1.9.17 archimedes.0.4.19 archsat.1.1 argot.1.1 asetmap.0.8.1 assertions.0.1 astring.0.8.5 async_core.109.55.02 atd2cconv.0.0.0 avroc.0.0.1 BetterErrors.0.0.1 b0.0.0.5 baardskeerder.0.5.2 bamboo.0.0.03 bap-build.2.5.0 base58.0.1.2 bau.0.0.4 bddapron.2.3.3 bear.0.0.1 bench.1.3 bes.0.9.4.2 bisect.1.3.1 bisect_ppx-ocamlbuild.1.0.1 bolt.1.4 bookaml.4.0 boomerang.1.1.0 bos.0.2.1 bpf.0.2 bracetax.0.4 brozip.1.1 brr.0.0.6 bson.0.89.3 bt.0.2 Camldiets.0.2 camldm.0.1.0 camlhighlight.5.0 camllib.1.3.3 camlp4.5.1 caper.0.9 captureio.0.1.2 caradoc.0.3 caravan.0.0.2 ccss.1.6 cgroups.0.1 channel.1.1.1 cil.1.7.3 cmark.0.2.0 cmarkit.0.3.0 coclobas.0.0.2 commonjs_of_ocaml.0.1.0 comparelib.113.00.00 cookie-js.1.0.0 corecount.1.0.0 cppo_ocamlbuild.1.6.9 crdt-ml.0.10.0 csvprovider.1.2 ctypes-build.0.0.2 custom_printf.113.00.00 DAGaml.0.02 deriving-yojson.0.4 diy.5.01 dnscurve.0.3.0 dog.0.2.1 dose.3.4.2 down.0.3.0 dyntype.0.9.0 earley-ocaml.1.1.0 efl.1.24.0 elf2json.1.0.0 emoji.1.1.0 enumerate.111.08.00 enumerators.0.2.0 erm_xml.0.3 erm_xmpp.0.2 estring.1.3 expect.0.0.6 ezirmin.0.2.1 facebook-sdk.0.3.5 faillib.111.17.00 farfadet.0.3 flock.1.0.0 flowtype.0.94.0 fmt.0.9.0 forkwork.0.3.2 fpath.0.7.3 frag.0.1.0 froc.0.2.2 fstar.2022.01.15 fstreams.0.2.1 fury-puyo.0.5 future.0.2.0 GuaCaml.0.05 gadelac.0.6.1 gamepad.0.2.0 gdb.0.3 gedcom.1.0.0 gen-bs.0.1.0 geoip.0.0.3 gg.1.0.0 grain_wasm_spec.0.1 grib.0.11.0 gsasl.0.10.0 gtk-light.0.0.1 hamt.1.0.0 hardcaml-affirm.0.1.0 hardcaml-bloop.0.1.0 hardcaml-llvmsim.0.3.0 hardcaml-vpi.0.3.0 hardcaml-waveterm.0.2.0 hardcaml-yosys.0.1.0 hdf.0.9.1 hdfs.0.4 heptagon.1.05.00 herelib.112.35.00 hevea.2.36 higher.0.3.1 hlarp.0.0.3 hmap.0.8.1 htmlit.0.1.0 http2https.1.0.0 humane-re.0.1.1 i2c.0.1.2-0 ibx.0.8.1 imap.1.1.1 imaplet-lwt.0.1.15 indexmap.0.0.3 io.0.2.0 iocaml.0.4.9 iocaml-kernel.0.4.8 irrlicht.0.0.5 iso-filesystem.0.1 jane-street-tests.v0.10.0 javascriptcore.0.0.1 joolog.0.4 js-build-tools.113.33.04 js-lz4.109.38.alpha1 js_of_ocaml-ocamlbuild.5.0 json-pointer.0.1.1-0 json-predicate.0.2.0-0 json-rpc.0.5.0-0 jsonm.1.0.2 jsoo_router.1.0 KaSim.4.0.0 kaputt.1.2 karamel.1.0.0 ketrew.3.2.1 krb5.109.38.alpha1 lacc.0.2 lambdoc.1.0-beta4 lem.2022-12-10 libra-tk.1.1.2 libres3.1.3 libssh.0.1 libudev.0.2.1 libvhd.0.9.0 lilis.0.2.1 linkage.0.1 linksem.0.8 lipsum.0.2 litiom.4.0 llvmgraph.0.2 logs.0.7.0 lwt-binio.0.2.1 lwt-zmq.2.1.0 lymp.0.2.5 lzo.0.0.2 macaque_lwt.0.1 macaroons.0.1.1 maildir.0.9.0 maki.0.2 mariadb.1.1.6 mascot.1.0 melt.1.4.0 memcpy.0.2.2 merge-queues.0.2.0 merge-ropes.0.2.0 mergeable-vector.0.1.0 mezzo.0.0.m8 minima-theme.1.0.0 mirage-btrees.0.1.0 mirage-clock-xen.1.1 mirage-entropy-xen.0.3.0 mirage-http-unix.1.0.0 mirage-http-xen.1.0.0 mirage-net-direct.0.9.4 mirage-net-socket.0.9.4 mirage-os-shim.3.1.0 mirage-seal.0.4.2 mirage-tc.0.3.0 mirage-tcpip-unix.0.9.5 mirage-tcpip-xen.0.9.5 mirror.0.0.1 mlcuddidl.3.0.7 mld.0.2 modelica_ml.0.2.0 monadlib.0.2 mongo.0.67.2 mosquitto.0.2.1-0 mqtt_client.0.0.1 msgpack.1.3.0 mtime.2.0.0 named-pipe.0.4.0 namespaces.0.5.1 nanomsg.1.0 nebula.0.2.1 netml.0.1.0 nlp.0.0.1 nocrypto.0.5.4-2 nosetup.0.8.1 note.0.0.3 oasis.0.4.11 oasis-mirage.0.3.0a oasis2debian.0.1.5 oasis2opam.0.8 obandit.0.3.4 obeanstalk.0.1 objsize.0.18 obytelib.1.6 ocaml-arg.0.3 ocaml-buddy.0.6.1 ocaml-data-notation.0.0.11 ocaml-migrate-parsetree-ocamlbuild.1.2.0 ocaml-protoc-yojson.0.2.0 ocaml-sat-solvers.0.7.1 ocamlbuild-atdgen.0.1.0 ocamlbuild-pkg.0.2.1 ocamlbuild-protoc.0.1 ocamlclean.2.3 ocamlify.0.0.1 ocamllint.0.3.0 ocamlmod.0.0.9 ocamlnet.4.1.9-2 ocamlpp.1.1 ocapic.3.5 ocb-stubblr.0.1.1-1 ocephes.0.8.2 oci.0.3 ocp-index-top.0.5.0 ocsfml.2.0 odig.0.0.9 odisco.0.1.3 ojquery.0.1 ojwidgets.0.1 olmi.1.1 oloop.0.1.2 oml.0.0.7 omod.0.0.4 omonad.0.4.0 oolc.0.3 opa-base.1.1.0+4263 opam-build-revdeps.0.1.0 opam-doc.0.9.3 opam-query.1.3 opam-sync-github-prs.1.1.0 openflow.0.9.1 operf-macro.0.2 oplay.1.0.0 optcomp.1.6 orm.0.7.1 orocksdb.0.4.0 osm_xml.0.0.1 ospec.0.3.2 ostap.0.5 otags.4.01.1 otfm.0.4.0 ox.1.1.1 pa_bench.113.00.00 pa_bin_prot.113.00.01 pa_comprehension.0.4 pa_do.0.8.16 pa_fields_conv.113.00.01 pa_ounit.113.00.00 pa_sexp_conv.113.00.01 pa_solution.0.7 pa_structural_sexp.113.00.00 pa_test.112.24.00 pa_typerep_conv.113.00.01 pa_variants_conv.109.15.04 pa_where.0.4 packet.0.4.0 pareto.0.3 pcf-format.0.0.1 pci.1.0.1 pci-db.0.3.0 perf.1.0 pgsolver.4.1 pipebang.113.00.00 plist.1.0.0 polyglot.1.0.0 posix-clock.0.4.0-0 posix-math.0.3.1-0 posix-mqueue.0.8.0-0 posix-semaphore.0.5.0-0 posix-time.1.0.1-0 ppx_debugger.1.0 ppx_deriving_argparse.0.0.5 ppx_deriving_crowbar.0.1.1 ppx_measure.1.1 process.0.2.1 procord.0.2.0 profound.0.4.2 proj4.0.9.2 promela.0.4.2 ptime.1.1.0 qcstm.0.1.1 qfs.0.12 qocamlbrowser.0.2.12 qrc.0.1.0 quickcheck.1.0.2 ranger.0.1.0 raygun4ocaml.0.1.0 rdr.2.0.1 react.1.2.2 receive-mail.0.1.2 regstab.2.0.0 release.1.1.1 riak.1.0.0 riak-pb.1.0.0 rlp.0.1 rotor.0.1 rresult.0.7.0 rtime.0.9.3 Snowflake.0.02.03 safa.1.4 sattools.0.1.0 scid.1.0 sendmsg.0.0.2 sequoia.0.1.0 sessions.0.1.0 setr.0.1.1 shcaml.0.2.1 should.0.1.0 sibylfs-lem.0.4.0 sid.2.0 sill.1.4 simple-bmc.0.0.1 simple-diff.0.3 simple_pam.0.1.2 skkserv-lite.2.0.1 smtp.0.3 sociaml-facebook-api.0.4.1 sociaml-oauth-client.0.5.0 sociaml-tumblr-api.0.2.0 sociaml-vcard.0.2.1 socketcan.0.8.12-0 sodium.0.6.0 solvuu-build.0.3.0 solvuu_build.0.0.2 sonet.0.1.2 sosa.0.3.0 spatial_index.0.0.2 spotify-cli.0.3.0 spreadsheet.0.1 sqlite3EZ.0.1.0 starterkit.1.0.0 statmemprof-emacs.0.1.0 statverif.1.97pl1.2 stemming.0.2.1 swdogen.0.1.0 symkat.1.4 TCSLib.0.3 tar-format.0.7.1 tdk.0.2.0 testsimple.0.3.1 text.0.8.1 tftp.0.1.4 tgls.0.8.6 themoviedb.0.8.1 tlstunnel.0.2.0 topkg.1.0.7 topkg-care.1.0.7 topology.0.4.0 tplib.1.3 tptp.0.3.2 travis-senv.1.0.0 tsdl.1.0.0 ttweetnacl.0.1.0 type-beat.0.1 type_conv.113.00.02 typehashlib.109.15.04 uchar.0.0.2 udunits.0.2.0 ulex.1.2 ulex-camlp5.1.3 unix-dirent.0.3.5 unix-fcntl.0.3.5 unix-sys-resource.0.1.2 unix-sys-stat.0.5.2 unix-time.0.1.0 unix-type-representations.0.1.2 unix-unistd.0.4.0 usb.1.3.1 usbmux.1.3.2 user-setup.0.7 uucd.15.1.0 uucp.15.1.0 uuidm.0.9.8 uunf.15.1.0 uuseg.15.1.0 uutf.1.0.3 vg.0.9.5 vhdlib.0.9.1 volt.1.4 wasm.2.0.1 webbrowser.0.6.1 webidl.1.4 wiringpi.0.0.1 xapi-idl.1.14.0 xe.0.6.3 xe-unikernel-upload.0.5 xen-block-driver.0.2.5 xen-disk.1.2.1 xenbigarray.1.0.0 xenctrl.0.10.0 xentropyd.0.9.3 xmlm.1.4.0 xxhash.0.1 yajl.0.7.3 yajl-extra.0.7.3 yices2_bindings.0.2 zbar.0.9 zephyrus.0.1 zipc.0.2.0 ```

gasche commented 2 months ago

I was going to ask whether we can test only direct dependencies, not transitive dependencies. But in fact it looks like opam lists only direct dependencies, and needs --recursive to also list transitive dependencies.

In addition to not testing packages that moved to dune in their later version, I think that a reasonable extra requirement to have would be to restrict to packages whose last release is at most N years old. It may be easier to write a script to precompute this list, and maybe think of refreshing it from times to times (it could be in our howto-release notes) than to compute it dynamically

gasche commented 2 months ago

(Hm: process, last released in 2016, is a dependency of karamel, released in 2023. So maybe my idea isn't so good.)

hhugo commented 2 months ago

https://github.com/ocaml/ocamlbuild/pull/348

ocaml / ocamlbuild

Improve testing: the ci should install more packages that depend on ocamlbuild #346