search

JuliaLang / julia

The Julia Programming Language
https://julialang.org/
MIT License
45.89k stars 5.49k forks source link

`]dev AnyStdlib` fails #51374

Open LilithHafner opened 1 year ago

LilithHafner commented 1 year ago 
(@v1.11) pkg> dev Dates
    Updating registry at `~/.julia/registries/General.toml`
ERROR: The following package names could not be resolved:
 * Dates (ade2ca70-3891-5945-98fb-dc099432e06a in manifest but not in project)

(@v1.11) pkg> dev Unicode
ERROR: The following package names could not be resolved:
 * Unicode (4ec0a83e-493e-50e2-b9ac-8f72acf5a8f5 in manifest but not in project)

(@v1.11) pkg> dev SortingAlgorithms
   Resolving package versions...
    Updating `~/.julia/environments/v1.11/Project.toml`
  [a2af1166] + SortingAlgorithms v1.1.1 `~/.julia/dev/SortingAlgorithms`
    Updating `~/.julia/environments/v1.11/Manifest.toml`
  [34da2185] + Compat v4.9.0
  [864edb3b] + DataStructures v0.18.15
  [a2af1166] + SortingAlgorithms v1.1.1 `~/.julia/dev/SortingAlgorithms`

This is a self contained, actionable, subset of #35991. Getting ]free to work would be another bonus.

KristofferC commented 1 year ago

What should dev Unicode do exactly?

LilithHafner commented 1 year ago

Just like ]dev SortingAlgorithms, it should 1) make the latest source code of the Unicode package available in the folder joinpath(Pkg.devdir(), "Unicode") 2) make it so that loading Unicode (e.g. using Unicode) loads the code in that folder 2.5) If Revise is loaded prior to the new instance of Unicode, make sure that changes made to the source code in joinpath(Pkg.devdir(), "Unicode") are tracked by revise 3) make it so that I can run git checkout -b my-new-branch/git commit -a -m "my changes"/gh pr create and contribute my changes upstream. 3.5) Make it possible to ]dev Unicode and ]dev Dates without having git combine the changes to the two codebases (just like how ]dev SortingAlgorithms and ]dev Plots don't get combined)

To support steps 3 and 3.5 for Unicode (which does not have its own Repo) we can clone julia into joinpath(Pkg.devdir(), ".Unicode") and create the symlink symlink(joinpath(Pkg.devdir(), ".Unicode", "stdlibs", "Unicode"), joinpath(Pkg.devdir(), "Unicode")).

LilithHafner commented 1 year ago

"Sparse checkout" may help make this look more pretty, but I don't think it will actually save bandwidth or disk space.

KristofferC commented 1 year ago

make it so that loading Unicode (e.g. using Unicode) loads the code in that folder

Cannot really happen since Unicode is in the sysimage and you can only load one instance of a package.

LilithHafner commented 1 year ago

We could automate the UUID changing process described here to trick ourselves into loading a new version of the stlib for direct use, but without displacing the existing version that other packages depend on.

KristofferC commented 1 year ago

That sounds very weird then that your dependencies do not load the package that you devved (which is unlike the case for any other normal package).

LilithHafner commented 1 year ago

That's true :/. Do you have any ideas for how we could support developing standard libraries which remain in the sysimg?

Setting the Revise tracking target to a new codebase seems inadequate because we can't redefine types that way.

Perhaps it is possible to reload packages in the sysimg the same way it is possible to reload packages defined in Main?