search

mxcube / mxcubeweb

MXCuBE-Web
http://mxcube.github.io/mxcubeweb/
GNU Lesser General Public License v3.0
21 stars 38 forks source link

The latest mxcubeweb is never tested against the latest mxcubecore #1371

Open fabcor-maxiv opened 1 month ago

fabcor-maxiv commented 1 month ago

The way our GitHub Actions workflows (CI pipelines) are setup the latest mxcubeweb is tested against the currently pinned version of mxcubecore.

This leads to incompatibility issues between mxcubecore and mxcubeweb to remain undetected for some time...

See this line, where it shows that we install with Poetry which means the poetry.lock file is taken into account and it is the pinned version of mxcubecore that is installed for the tests:

https://github.com/mxcube/mxcubeweb/blob/e0b3ab76d015ba5c97121313da601c1be531d5c2/.github/workflows/build_and_test.yml#L53

Here in mxcubeweb, the currently pinned version of mxcubecore is 1.133.0:

https://github.com/mxcube/mxcubeweb/blob/e0b3ab76d015ba5c97121313da601c1be531d5c2/poetry.lock#L1750-L1759

The current latest tagged version of mxcubecore is 1.142.0:

https://github.com/mxcube/mxcubecore/blob/61b7828cac708f17b4039ebd3a0632c477a1ace4/pyproject.toml#L3

So currently the mxcubeweb CI pipelines test against a quite outdated version of mxcubecore.

Maybe we should look into this, investigate how bad the damage could be, think of processes and technical solutions to minimize undetected issues.

fabcor-maxiv commented 1 month ago

Examples of issues that might have been detected:

marcus-oscarsson commented 1 month ago

How about using ^ in the dependency list ?

fabcor-maxiv commented 1 month ago

How about using ^ in the dependency list ?

Would not change anything because it is still the version pinned in poetry.lock that would be installed.

axelboc commented 1 month ago

Saw this recently: https://docs.astral.sh/uv/concepts/workspaces/ — not a drop-in solution, obviously...

marcus-oscarsson commented 1 month ago

How about using ^ in the dependency list ?

Would not change anything because it is still the version pinned in poetry.lock that would be installed.

Indeed, it would mean that we have to re-generate poetry.lock. Would it make sense to test against the pinned version and create sort of a "smoke" test for the latest version in a separate pipeline ?

fabcor-maxiv commented 1 month ago

There is a lot of aspects to consider in this topic. I do not believe there is one quick and easy solution. I also do not believe that we need to aim for perfect airtight solution. I believe maybe we can minimize such issues by doing some careful changes in tooling and process.

Anyway, one aspect of this, is that in principle, for each tagged version in CORE we should check that it does not break WEB. If it breaks WEB, then this version of CORE should be excluded from the valid dependencies range.

Would it make sense to test against the pinned version and create sort of a "smoke" test for the latest version in a separate pipeline ?

Yes, something along those lines would probably help. This is probably a very good ratio effort vs. value.

Maybe important is to decide what we want to achieve, what rules are we setting for ourselves. Do we want to set as a policy that latest tagged WEB should work with latest tagged CORE? Or something else?

Saw this recently: https://docs.astral.sh/uv/concepts/workspaces/ — not a drop-in solution, obviously...

Yes, maybe that could help. But I would be careful about those new Python dev workflow tools (uv, rye, hatch, pdm, and so on), they are all somewhat new and have lots of breaking changes. I feel like Poetry is just now finally stable enough for a long enough time : D

rhfogh commented 1 month ago

One approach would be to always test against current web and current core versions. If you are making changes in tandem you might want to specify a specific version combination but otherwise, if current web does not match current core then surely there is something wrong?

axelboc commented 1 month ago

I'm pretty sure it's possible to trigger workflows across repositories. Let's dream big:

  1. Open PR for my-branch in mxcubecore
  2. Trigger a workflow W1 in mxcubeweb.
  3. W1 checks out my-branch of mxcubecore and installs it from source.
  4. W1 runs all mxcubeweb tests and reports back with a comment in the PR on mxcubecore.
  5. Fix my-branch if it broke mxcubeweb unexpectedly, or simply ignore the breakage if it's expected.
  6. Merge my-branch into main and wait for tag to be created.
  7. Trigger workflow W2 in mxcubeweb, now testing against the latest tag of mxcubecore.
  8. If W2 fails, open a PR in mxcubeweb to upgrade mxcubecore to the latest tag.
marcus-oscarsson commented 1 month ago

@rhfogh: Yes indeed current web should always work with latest core.

marcus-oscarsson commented 1 month ago

@axelboc: yes thats a nice idea :) seems like a reasonable and doable test