Naming a package extension "REPLExt" breaks loading

[tecosaur]

So, I've been working on decreasing TTFX in a package of mine, and thought in the process I'd split the REPL functionality up into a package extension ... called REPLExt.

Turns out this causes issues though, do package extension names need to be globally unique? That would be weird, but it kinda seems like it from this MWE.

pkg_ext_name_repro.zip

This is just the results of ]generate pkg_ext_name_repo + a minimal REPL-based package extension.

Reproduction instructions:

• Extract (and instantiate?) pkg_ext_name_repo
• Run julia +1.11 --startup-file=no --project=pkg_ext_name_repo
• Try using Pkg
• Observe the below error message

julia> using Pkg
[ Info: Precompiling REPLExt [e5eb5ef1-03cf-53a7-ae1d-5a66b08e832b] (cache misses: wrong dep version loaded (2), wrong source (2), invalid header (8), mismatched flags (2))
ERROR: LoadError: ArgumentError: Package REPLExt does not have pkg_ext_name_repro in its dependencies:
- Note that the following manifests in the load path were resolved with a different
  julia version, which may be the cause of the error:
    /home/tec/pkg_ext_name_repro/Manifest.toml (vnothing)
- You may have a partially installed environment. Try `Pkg.instantiate()`
  to ensure all packages in the environment are installed.
- Or, if you have REPLExt checked out for development and have
  added pkg_ext_name_repro as a dependency but haven't updated your primary
  environment's manifest file, try `Pkg.resolve()`.
- Otherwise you may need to report an issue with REPLExt
Stacktrace:
 [1] macro expansion
   @ ./loading.jl:2133 [inlined]
 [2] macro expansion
   @ ./lock.jl:273 [inlined]
 [3] __require(into::Module, mod::Symbol)
   @ Base ./loading.jl:2105
 [4] #invoke_in_world#3
   @ ./essentials.jl:1064 [inlined]
 [5] invoke_in_world
   @ ./essentials.jl:1061 [inlined]
 [6] require(into::Module, mod::Symbol)
   @ Base ./loading.jl:2098
 [7] include
   @ ./Base.jl:558 [inlined]
 [8] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::Nothing)
   @ Base ./loading.jl:2721
 [9] top-level scope
   @ stdin:4
in expression starting at /home/tec/pkg_ext_name_repro/ext/REPLExt.jl:1
in expression starting at stdin:4
┌ Error: Error during loading of extension REPLExt of Pkg, use `Base.retry_load_extensions()` to retry.
│   exception =
│    1-element ExceptionStack:
│    Failed to precompile REPLExt [e5eb5ef1-03cf-53a7-ae1d-5a66b08e832b] to "/home/tec/.julia/compiled/v1.11/REPLExt/jl_f6PsaY".
│    Stacktrace:
│      [1] error(s::String)
│        @ Base ./error.jl:35
│     ...
│     [41] _start()
│        @ Base ./client.jl:536
└ @ Base loading.jl:1437

[IanButterworth]

What if the name isn't REPLExt but the same name as a non stdlib ext?

[KristofferC]

Probably some interaction with require_stdlib

[tecosaur]

FYI: I think I'm hitting the same issue with two packages that both define PkgExt, is that also part of a stdlib?

[IanButterworth]

Is your manifest up to date? The error message mentions it.

[KristofferC]

I think I'm hitting the same issue with two packages that both define PkgExt, is that also part of a stdlib?

No. Hm, I don't think the loading code should ever identify a package only by name and they should get different UUIDs because their parent is different. But if course, might be a bug somewhere. But I'm pretty sure there are many exte sions in the ecosystem with the same name so I would have thought that would have already showed up if it was an issue.

[aplavin]

Extensions with the same name definitely work, and I think there was no released Julia version where they didn't. You can easily have a whole bunch, not just two:

julia> using UnicodePlots, SkyCoords, Normalization, PlutoTables, FlexiMaps, VOTables, AxisKeysExtra, PyPlotUtils, Unitful

# there are 8 UnitfulExts here:
(jl_RKhdWo) pkg> st --extensions
Status `/private/var/folders/2j/9vtd991d201dbkh9dnx3wvy00000gq/T/jl_RKhdWo/Project.toml`
  [b7a0d2b7] AxisKeysExtra v0.1.10
              ├─ RectiGridsExt [RectiGrids]
              ├─ UnitfulExt [Unitful]
              ├─ DimensionalDataExt [DimensionalData]
              └─ MakieExt [Makie]
  [6394faf6] FlexiMaps v0.1.26
              ├─ IntervalSetsExt [IntervalSets]
              ├─ AxisKeysExt [AxisKeys]
              ├─ DictionariesExt [Dictionaries]
              ├─ StructArraysExt [StructArrays]
              └─ UnitfulExt [Unitful]
  [be38d6a3] Normalization v0.7.1
              ├─ DataFramesExt [DataFrames]
              ├─ UnitfulExt [Unitful]
              └─ DimensionalDataExt [DimensionalData]
  [e64c0356] PlutoTables v0.1.6
              └─ UnitfulExt [Unitful]
  [5384e752] PyPlotUtils v0.1.31
              ├─ AxisKeysExt [AxisKeys]
              ├─ AxisKeysUnitfulExt [AxisKeys, Unitful]
              └─ UnitfulExt [Unitful]
  [fc659fc5] SkyCoords v1.4.0
              ├─ AccessorsExt [Accessors]
              └─ UnitfulExt [Unitful]
  [b8865327] UnicodePlots v3.6.4
              ├─ IntervalSetsExt [IntervalSets]
              ├─ TermExt [Term]
              ├─ FreeTypeExt [FileIO, FreeType]
              ├─ UnitfulExt [Unitful]
              └─ ImageInTerminalExt [ImageInTerminal]
  [1986cc42] Unitful v1.20.0
              ├─ InverseFunctionsUnitfulExt [InverseFunctions]
              └─ ConstructionBaseUnitfulExt [ConstructionBase]
  [3abbdab7] VOTables v0.1.14
              ├─ DictArraysExt [DictArrays]
              └─ UnitfulExt [Unitful]

Maybe, your issue is somehow related to the deps being in stdlib? There are known issues for that scenario, even without duplicate names – eg https://github.com/JuliaLang/julia/issues/52132.

[KristofferC]

That's possible. Should be fixed in 1.11 I think it is.

[tecosaur]

Some more info/responses:

• I'm using Julia 1.11-beta2
• My manifest(s) are up to date, as best I know

In my global environment, here's where one PkgExt is defined:

(@v1.11) pkg> st --extensions
Status `~/.julia/environments/v1.11/Project.toml`
  [1e701ad1] Setup v0.1.0 `~/.julia/config/Setup`
              ├─ PkgExt [Pkg]
              ...

and then when the error appears, it's when I'm developing DataToolkitCommon which also has a PkgExt = "Pkg" extension.

[disberd]

I have also faced this in the last days dealing with multiple packages defining an extension for PlotlyBase (so definitely not a stdlib) called PlotlyBaseExt. Problems seems to be fixed if I changed the name of either of the two extensions called PlotlyBaseExt.

I didn't manage to narrow down an MWE but I'll post it if I manage to

[tecosaur]

Renaming my startup package's PkgExt to PkgUtils has resolved the issue with that name I mentioned earlier: https://code.tecosaur.net/tec/Setup.jl/commit/443552f2176f8ff00d564a58c25ba0cce5a96185

[KristofferC]

I can repro this so I should be able to understand what is going on.

[KristofferC]

https://github.com/JuliaLang/julia/pull/54658 should fix this.

[tecosaur]

Thanks Kristoffer!