search

JuliaRegistries / General

The official registry of general Julia packages
https://github.com/JuliaRegistries/General/blob/master/README.md
MIT License
610 stars 471 forks source link

The `eager` registry flavor is significantly lagging behind the `master` branch #120562

Open GeorgeR227 opened 18 hours ago

GeorgeR227 commented 18 hours ago

I've released a new version of CombinatorialSpaces.jl a few hours ago when writing this but it seems like that release hasn't made it to the General registry yet. Just to confirm the issue isn't with this specific release I've tried with a few other packages unrelated to this one and going back even further, even like 12 hours ago, I can't seem to update to the newest versions of those either.

There probably isn't, and can't be I suppose, a guarantee on how long this propagation can take but this feels like significantly longer than usual.

DilumAluthge commented 18 hours ago

If you need the latest package version right now, you can try switching to the "eager" flavor of the registry:

ENV["JULIA_PKG_SERVER_REGISTRY_PREFERENCE"] = "eager"

import Pkg

Pkg.Registry.update()

Try that out, and if you can't get the latest package version when you're using the eager registry, let me know, and I'll reopen this issue.

GeorgeR227 commented 18 hours ago

I tried running those commands but I still can't get the newest version to download. However, I've noticed that one of those other packages I tested before (Enzyme.jl specifically) that released a longer time ago seems to have finally propagated. In this case I'm expecting version 0.6.8 for CombinatorialSpaces.jl and Enzyme.jl is now on 0.13.18.

julia> ENV["JULIA_PKG_SERVER_REGISTRY_PREFERENCE"] = "eager"
"eager"

julia> import Pkg

julia> Pkg.Registry.update()
    Updating registry at `~/.julia/registries/General.toml`

(@v1.11) pkg> add CombinatorialSpaces
   Resolving package versions...
    Updating `~/.julia/environments/v1.11/Project.toml`
  [b1c52339] + CombinatorialSpaces v0.6.7

...

(@v1.11) pkg> add Enzyme
   Resolving package versions...
   Installed Enzyme ─ v0.13.18
    Updating `~/.julia/environments/v1.11/Project.toml`
  [7da242da] + Enzyme v0.13.18
GeorgeR227 commented 17 hours ago

If this is simply a case of "sit and wait", that's not too much of a problem for me. However, I just wanted to point this out in case there was some kind of unintended slowdown/blockage in the registry.

DilumAluthge commented 17 hours ago

Well, the goal of the eager registry is to never have that delay. So this may be a bug in the Pkg server.

NicolasRiel commented 9 hours ago

Same issue on my side, been waiting 12 hours, also using "Eager"

DilumAluthge commented 9 hours ago

I am able to reproduce this in a clean setup with Julia 1.11.1. Here are the steps to reproduce.

First, run the following commands in Bash:

unset JULIA_NUM_THREADS

unset JULIA_PROJECT

unset JULIA_LOAD_PATH

unset JULIA_PKG_SERVER

export JULIA_DEPOT_PATH="$(mktemp -d)"

julia --startup-file=no --history-file=no

Now, inside Julia, run the following commands:

ENV["JULIA_PKG_PRECOMPILE_AUTO"] = "0"

ENV["JULIA_PKG_SERVER_REGISTRY_PREFERENCE"] = "eager"

import Pkg

Pkg.Registry.add()

Pkg.Registry.update()

Pkg.add("CombinatorialSpaces")

Pkg.status()

Pkg.add(; name = "CombinatorialSpaces", version = "0.6.8")

Here is the output that I get:

$ unset JULIA_NUM_THREADS
$ unset JULIA_PROJECT
$ unset JULIA_LOAD_PATH
$ unset JULIA_PKG_SERVER
$ export JULIA_DEPOT_PATH="$(mktemp -d)"
$ julia --startup-file=no --history-file=no
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.11.1 (2024-10-16)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> ENV["JULIA_PKG_PRECOMPILE_AUTO"] = "0"
"0"

julia> ENV["JULIA_PKG_SERVER_REGISTRY_PREFERENCE"] = "eager"
"eager"

julia> import Pkg
[ Info: Precompiling Pkg [44cfe95a-1eb2-52ea-b672-e2afdf69b78f]

julia> Pkg.Registry.add()
  Installing known registries into `/var/folders/f4/dd0zhsvj5099rqhl7v7wjq580000gq/T/tmp.wOmv6COwdI`
       Added `General` registry to /tmpdepot/registries
true

julia> Pkg.Registry.update()
    Updating registry at `/tmpdepot/registries/General.toml`

julia> Pkg.add("CombinatorialSpaces")
    Updating registry at `/tmpdepot/registries/General.toml`
   Resolving package versions...

...

    Updating `/private/tmpdepot/environments/v1.11/Project.toml`
  [b1c52339] + CombinatorialSpaces v0.6.7
    Updating `/private/tmpdepot/environments/v1.11/Manifest.toml`

...

julia> Pkg.status()
Status `/private/tmpdepot/environments/v1.11/Project.toml`
  [b1c52339] CombinatorialSpaces v0.6.7

julia> Pkg.add(; name = "CombinatorialSpaces", version = "0.6.8")
   Resolving package versions...
ERROR: Unsatisfiable requirements detected for package CombinatorialSpaces [b1c52339]:
 CombinatorialSpaces [b1c52339] log:
 ├─possible versions are: 0.1.0 - 0.6.7 or uninstalled
 ├─restricted to versions * by project [5aa5a18c], leaving only versions: 0.1.0 - 0.6.7
 │ └─project [5aa5a18c] log:
 │   ├─possible versions are: 0.0.0 or uninstalled
 │   └─project [5aa5a18c] is fixed to version 0.0.0
 └─restricted to versions 0.6.8 by an explicit requirement — no versions left
Stacktrace:

...

However, looking at the registry, I see that CombinatorialSpaces v0.6.8 should be available when using the eager registry:

https://github.com/JuliaRegistries/General/blob/4efa4f52673180858dd4c91b0c8df93c4c09f7cd/C/CombinatorialSpaces/Versions.toml#L64-L65

DilumAluthge commented 8 hours ago

I took a look at which commits registries.eager and registries.conservative currently point to. Here is what I get:

Latest commit on origin/master:
commit 4efa4f52673180858dd4c91b0c8df93c4c09f7cd
Author: Registrator <ci@juliacomputing.com>
Date:   Tue Dec 3 09:55:54 2024 +0530
    New version: GasChem v0.8.0 (#120574)

Eager commit:
commit 7aa56ada7e91056215ab4e18a37ba7f6081f0b0f
Author: Registrator <ci@juliacomputing.com>
Date:   Mon Dec 2 15:14:01 2024 +0530
    New version: PkgSkeleton v1.3.2 (#120518)

Conservative commit:
commit 6ec563265b0dcff3a3fbe9b9445e6523de125801
Author: Registrator <ci@juliacomputing.com>
Date:   Mon Dec 2 10:39:24 2024 +0530
    New version: Gridap v0.18.8 (#120508)

In hyperlink format:

DilumAluthge commented 8 hours ago

So currently, registries.eager is not giving the latest commit.