choosenim still doesn't use `git fetch`

timotheecour commented 4 years ago

@genotrance

[ ] https://github.com/dom96/choosenim/issues/12 was closed after merging https://github.com/dom96/choosenim/pull/173 however it still seems to do a download instead of a fetch:

C:\Users\timothee\git_clone\choosenim>choosenim -v
choosenim v0.6.0 (2020-03-06 18:45:17) [windows/amd64]

on windows choosenim update --verbose #devel shows:

choosenim update --verbose #devel
  Info: No client ID found in 'C:\Users\timothee\.choosenim\analytics', not sending analytics.
  Info: No client ID found in 'C:\Users\timothee\.choosenim\analytics', not sending analytics.
Updating #devel
Downloading Nim devel from GitHub
[##################################################] 100.0% 0kb/s
  Info: No client ID found in 'C:\Users\timothee\.choosenim\analytics', not sending analytics.
Extracting nim-1.3.5-windows_x64.zip
Setting up git repository
  Info: No client ID found in 'C:\Users\timothee\.choosenim\analytics', not sending analytics.
Building Nim #devel
Compiler: Already built
 Tools: Already built
  Info: No client ID found in 'C:\Users\timothee\.choosenim\analytics', not sending analytics.
  Info: No client ID found in 'C:\Users\timothee\.choosenim\analytics', not sending analytics.
Updated to #devel

and re-running re-downloads; I thought it would instead cause git fetch or similar instead of having to redownload a zip

likewise on OSX:

choose update "#devel"
   Updating #devel
Downloading Nim devel from GitHub
[##################################################] 100.0% 0kb/s
 Extracting nim-1.3.3-osx.tar.xz
    Setting up git repository
   Building Nim #devel
   Building koch
   Building Nim
   Building tools (nimble, nimgrep, nimpretty, nimsuggest, testament)
    Updated to #devel

and then re-running re-does the whole build

this is how I'd intend this feature to work: choosenim update #devel:

git fetch
if remotes/head matches a hash we've already built, nothing to do
else, checkout that hash and build from it; no zip download needed

also:

[ ] what's the difference between choosenim update #head and choosenim update #devel (other ppl were wondering too)

genotrance commented 4 years ago

Choosenim recognizes #head and devel is a channel. Both can be updated.

#devel is treated as a commit so its deleted and rebuilt from scratch.

timotheecour commented 4 years ago

not sure whether you're implying it works as intended or not; but I don't see where fetch (from https://github.com/dom96/choosenim/pull/173) happens, it always seems to use download a zip.

devel is treated as a commit so its deleted and rebuilt from scratch.

but it's the same with choosenim update #head

there are 2 options:

option 1:

resolve symbolic hashes (eg #head) the real hash (after a fetch you know the hash).

instead of saving to ~/.choosenim/toolchains/nim-#head/ it should save to ~/.choosenim/toolchains/nim-c61f7466292cb19c910f4a2f2976f738796e6bdf and on systems that use symlinks (eg posix + windows 10), use a symlink ln -s nim-#head nim-c61f7466292cb19c910f4a2f2976f738796e6bdf, or simply store this relationship in a json file.

likewise for nimble, nimble install choosenim@#head should install to ~/.nimble/pkgs/choosenim-0a6a7933da9e532679998000fdc7aca4f1cec643 instead of ~/.nimble/pkgs/choosenim-\#head/

this allows keeping more than 1 head (eg after a few updates) but requires some form of garbage collection (as done in homebrew)

option 2 (simplest but only allows keeping latest hash, which may be ok):

keep symbolic hashes, eg ~/.choosenim/toolchains/nim-#head/ and ~/.nimble/pkgs/choosenim-\#head/, but store relationship in a json file eg: ~/.nimble/map.json and ~/.choosenim/map.json

so next time you run choosenim update #head, it'll know whether the hash is already there or not

genotrance commented 4 years ago

I see your confusion. If you look at choosenim -h:

 choosenim devel [--latest]
    Installs (if necessary) and selects the most recent nightly build of Nim.
    The '--latest' flag selects and builds the latest commit in the devel branch

choosenim update devel [--latest]
    Updates to the most recent nightly build of Nim.
    The '--latest' flag updates and builds the latest commit in the devel branch

On Windows and Linux, it installs latest nightly release which is a binary build so you get "already built". On OSX, there's no binary builds yet so it rebuilds every time on update. But it is downloading an archive from nightlies/releases which is tested and known to be stable.

You only get git checkouts with --latest. This is how @dom96 wanted it to be. See #179 for the history.

genotrance commented 4 years ago

@timotheecour - is this good to close?

timotheecour commented 4 years ago

thanks for your explanation, the main thing missing here is to avoid rebuilding when un-necessary (because you've already built at that commit); right now re-running choosenim update devel --latest twice re-builds; the 2nd time should be noop

dom96 commented 4 years ago

How does that rebuild twice? Are you referring to building C sources and then bootstrapping? If so then that has to happen to get a reliable version built (in particular when building certain git commit hashes).

dom96 / choosenim