mamba-org / mamba

The Fast Cross-Platform Package Manager
https://mamba.readthedocs.io
BSD 3-Clause "New" or "Revised" License
6.97k stars 359 forks source link

Timestamp accuracy is reduced #3523

Open wolfv opened 1 month ago

wolfv commented 1 month ago

Sometimes mamba / libsolv cannot decide what package to choose and falls back to looking at the timestamp of the packages.

This is currently the case on Windows for the vc package.

The timestamp is however seen as equal in libsolv, while in truth it's different between the packages:

info     libsolv  Fallback to timestamp comparison: 1728400826 vs 1728400826: [-1]
info     libsolv  Selecting variant [a] of (a) vc-14.3-h8a93ad2_22 vs (b) vc-14.3-ha32ba9b_22 (score: -1)

I think this comes from a division by 1000 in libsolv: https://github.com/openSUSE/libsolv/blob/081580de9f964aae61d25e1787c37c7ab8b4fa69/ext/repo_conda.c#L302-L304

Which removes the necessary accuracy for this comparison. Maybe we can fix that in libsolv?! :)

{
  "key": "vc-14.3-h8a93ad2_22.conda",
  "value": {
    "build": "h8a93ad2_22",
    "build_number": 22,
    "depends": [
      "vc14_runtime >=14.40.33810"
    ],
    "license": "BSD-3-Clause",
    "license_family": "BSD",
    "md5": "a47cd756e88d8a80dfae678842d4acc9",
    "name": "vc",
    "sha256": "b72e7410ec0a748d21e9e997234fc474e3367cee8f509b07eb8182d3584a38d8",
    "size": 17462,
    "subdir": "win-64",
    "timestamp": 1728400826503,
    "track_features": "vc14",
    "version": "14.3"
  }
}
{
  "key": "vc-14.3-ha32ba9b_22.conda",
  "value": {
    "build": "ha32ba9b_22",
    "build_number": 22,
    "depends": [
      "vc14_runtime >=14.38.33135"
    ],
    "license": "BSD-3-Clause",
    "license_family": "BSD",
    "md5": "311c9ba1dfdd2895a8cb08346ff26259",
    "name": "vc",
    "sha256": "2a47c5bd8bec045959afada7063feacd074ad66b170c1ea92dd139b389fcf8fd",
    "size": 17447,
    "subdir": "win-64",
    "timestamp": 1728400826998,
    "track_features": "vc14",
    "version": "14.3"
  }
}
jjerphan commented 1 month ago

@mlschroe: do you remember the reason for performing this division here? Is the comparison of timestamps using another precision?

wolfv commented 1 month ago

Yes, there are (or used to be?) different accuracy timestamps around. I think in particular, Windows uses a less accurate timestamp by default. So this division is a normalization step (we also do something like that in rattler).

But now I think a multiplication would be better :)

wolfv commented 1 month ago

Conda: https://github.com/conda/conda/blob/8af4cdb2ab1c3e78116456afc445c4e6c8e4c369/conda/models/records.py#L61-L77

Rattler: https://github.com/conda/rattler/blob/3d86dba3f9e3aa6c5263d7b2e9f16285601c716c/crates/rattler_lock/src/utils/serde/timestamp.rs#L16-L20

Note: we should get rid of the serialization thing we do there.

jjerphan commented 1 month ago

One should probably try to first patch libsolv locally to test against libmamba.

mlschroe commented 1 month ago

SOLVABLE_BUILDTIME is defined to be the time in seconds, that's why the division is there. If you need sub-second resolution, we'll need to add a new SOLVABLE_BUILDTIME_NSEC key or something similar.

jjerphan commented 1 month ago

Thank you.

For reference, was the choice of second as resolution historically based on Windows'?

mlschroe commented 1 month ago

It's because both rpm and arch linux use "seconds since epoch" as build time in their package metadata.