Service is failing with `Build failed: Exception: Cancelled: Invalid_argument("filter_deps")`

[shonfeder]

Aug 21 08:37:28 arm64-jade-2 solver-service[1452342]: - Invalid_argument("filter_deps")
Aug 21 08:37:28 arm64-jade-2 solver-service[1452342]: - Invalid_argument("filter_deps")
Aug 21 08:37:28 arm64-jade-2 solver-service[1452342]:                                                            Called from Eio__core__Promise.await in file "lib_eio/core/promise.ml", lines 44-61, characters 4-7
Aug 21 08:37:28 arm64-jade-2 solver-service[1452342]:                                                            Raised at Stdlib__Effect.Deep.discontinue.(fun) in file "effect.ml", line 65, characters 41-48
Aug 21 08:37:28 arm64-jade-2 solver-service[1452342]: 2024-08-21 08:37.28  solver-worker [WARNING] Build failed: Exception: Cancelled: Invalid_argument("filter_deps")
Aug 21 08:37:28 arm64-jade-2 solver-service[1452342]: 2024-08-21 08:37.28  solver-worker [INFO] Requesting a new job...
Aug 21 08:37:28 arm64-jade-2 solver-service[1452342]: 2024-08-21 08:37.28      capnp-rpc [WARNING] Uncaught exception: Cancelled: Invalid_argument("filter_deps")
Aug 21 08:37:28 arm64-jade-2 solver-service[1452342]: 2024-08-21 08:37.28      capnp-rpc [WARNING] Uncaught exception: Cancelled: Invalid_argument("filter_deps")
Aug 21 08:37:28 arm64-jade-2 solver-service[1452342]:                                                            Called from Solver_worker.loop.loop.(fun) in file "worker/solver_worker.ml", line 98, characters 21-52
Aug 21 08:37:28 arm64-jade-2 solver-service[1452342]:                                                            Called from Solver_worker.build in file "worker/solver_worker.ml", line 61, characters 17-64
Aug 21 08:37:28 arm64-jade-2 solver-service[1452342]:                                                            Called from Solver_worker__Custom.solve in file "worker/custom.ml", lines 33-38, characters 4-62
Aug 21 08:37:28 arm64-jade-2 solver-service[1452342]:                                                            Called from Stdlib__Result.bind in file "result.ml" (inlined), line 23, characters 36-39
Aug 21 08:37:28 arm64-jade-2 solver-service[1452342]:                                                            Re-raised at Lwt_eio.run_lwt in file "lib/lwt_eio.ml", line 274, characters 4-12
Aug 21 08:37:28 arm64-jade-2 solver-service[1452342]:                                                            Called from Lwt_eio.run_lwt in file "lib/lwt_eio.ml", line 271, characters 4-23
Aug 21 08:37:28 arm64-jade-2 solver-service[1452342]:                                                            Called from Eio__core__Promise.await_exn in file "lib_eio/core/promise.ml", line 69, characters 8-15
Aug 21 08:37:28 arm64-jade-2 solver-service[1452342]:                                                            Called from Eio__core__Promise.await in file "lib_eio/core/promise.ml", lines 44-61, characters 4-7
Aug 21 08:37:28 arm64-jade-2 solver-service[1452342]:                                                            Raised at Stdlib__Effect.Deep.discontinue.(fun) in file "effect.ml", line 65, characters 41-48
Aug 21 08:37:28 arm64-jade-2 solver-service[1452342]: 2024-08-21 08:37.28  solver-worker [WARNING] Build failed: Exception: Cancelled: Invalid_argument("filter_deps")
Aug 21 08:37:14 arm64-jade-2 solver-service[1452342]: 2024-08-21 08:37.14  solver-worker [INFO] At capacity. Waiting for a build to finish before requesting more...
Aug 21 08:37:14 arm64-jade-2 solver-service[1452342]: 2024-08-21 08:37.14  solver-worker [INFO] Got request to build a job of kind "solve"
Aug 21 08:37:13 arm64-jade-2 solver-service[1452342]: 2024-08-21 08:37.13  solver-worker [INFO] Requesting a new job...
Aug 21 08:37:13 arm64-jade-2 solver-service[1452342]: 2024-08-21 08:37.13  solver-worker [INFO] Got request to build a job of kind "solve"
Aug 21 08:37:13 arm64-jade-2 solver-service[1452342]: 2024-08-21 08:37.13  solver-worker [INFO] Requesting a new job...
Aug 21 08:37:12 arm64-jade-2 solver-service[1452342]: 2024-08-21 08:37.12      capnp-rpc [INFO] Doing TLS client-side handshake...
Aug 21 08:37:12 arm64-jade-2 solver-service[1452342]: 2024-08-21 08:37.12      capnp-rpc [INFO] Connecting to ci3.ocamllabs.io:8103...
Aug 21 08:37:12 arm64-jade-2 solver-service[1452342]: 2024-08-21 08:37.12      capnp-rpc [INFO] Generated key with hash sha-256@ev55lMS9AeNH6tF_MpLLBT57BmyOisG0dqAEOuUkOdc
Aug 21 08:37:12 arm64-jade-2 solver-service[1452342]: 2024-08-21 08:37.12      capnp-rpc [INFO] Generating new private key...
Aug 21 08:37:12 arm64-jade-2 systemd[1]: Started Solver Service 1.
Aug 21 08:37:12 arm64-jade-2 systemd[1]: solver-service-1.service: Consumed 3min 12.665s CPU time.
Aug 21 08:37:12 arm64-jade-2 systemd[1]: Stopped Solver Service 1.
Aug 21 08:37:12 arm64-jade-2 systemd[1]: solver-service-1.service: Scheduled restart job, restart counter is at 7.
Aug 21 08:36:12 arm64-jade-2 systemd[1]: solver-service-1.service: Consumed 3min 12.665s CPU time.
Aug 21 08:36:12 arm64-jade-2 systemd[1]: solver-service-1.service: Failed with result 'exit-code'.
Aug 21 08:36:12 arm64-jade-2 systemd[1]: solver-service-1.service: Main process exited, code=exited, status=2/INVALIDARGUMENT

Given the timing, we assume it is related to the recent deployment of https://github.com/ocurrent/solver-service/pull/79 or its direct antecedents.

First noticed by @punchagan.

[mtelvers]

Initially, I rolled back to commit a11a91197a0382ca15b2d4e99abfbe8a0f5079b9 from 14th Aug 2024 but this has the same issue.

I am now using commit 48a157e5320ec0df707cf8616108cb998e12e681 from 25th Sept 2023 as this seems to be ok.

[talex5]

It's due to https://github.com/ocurrent/solver-service/pull/77, which updated to the dev version of opam. OpamFilter.filter_deps added a new optional ?dev_setup argument, and crashes if you don't pass it and a package uses {with-dev-setup}.

81 has a fix.