search

Mord3rca / gamma-launcher

Just another Launcher to setup S.T.A.L.K.E.R.: G.A.M.M.A.
GNU General Public License v3.0
84 stars 13 forks source link

Use git version control for GitHub downloads #125

Closed Singustromo closed 1 month ago

Singustromo commented 5 months ago

First of, thanks a bunch for still working on this.

I've got another proposal which I also initially expressed over on the gamma discord last year. Please let the user download repos from github via version control to save bandwidth and disk writes for updates. I personally download with 800KB/s tops and were able to place them in e.g. $gamma_dir/.Grok's Modpack Installer prior to the installer being so strict with existing files.

Also, shallow clones are a thing to save additional bandwith for the first download as the definition repo is over 9GB in size. git clone --depth 1 <...>

Mord3rca commented 4 months ago

Gonna work on that during the weekend.

Git repos will be in download folder instead of Grok's directory ... To take advantage of the CACHE option.

Singustromo commented 4 months ago

That's reasonable, thanks! Was just one thing that was missing. Downloading e.g. the large_files repo as a zip would take me over 3 hours when a new version is released x)

Mord3rca commented 4 months ago

A mirror will be used so a shallow clone is not possible (well ... it is but not a good solution)

This will be dl heavy at first and then you will only download the diff between your current HEAD and the remote HEAD.

PR is in review but not ready yet....

Mord3rca commented 1 month ago

Well, should be working now, can you give it a shot ? (on use_git_python branch)

The first DL will be long since you fetch all objects of the git archive

For reference:

$ du -hd0 *.git
3.1G    gamma_large_files_v2.git
4.0G    gamma_setup.git
1.6G    Stalker_GAMMA.git
574M    teivaz_anomaly_gunslinger.git
nlgranger commented 1 month ago

FYI, if the repo has many "large" files with multiple versions, cloning with "--filter=blob:none" will accelerate the initial clone by skipping older versions of the files. Contrary to shallow clones, this does not mess with the ability to fetch new commits or checkout in the history.

https://github.blog/open-source/git/get-up-to-speed-with-partial-clone-and-shallow-clone/

Mord3rca commented 1 month ago

Thanks for the doc @nlgranger , unfortunately git clone is not used and fetch / init do not support --filter option