search

sct-pipeline / contrast-agnostic-softseg-spinalcord

Contrast-agnostic spinal cord segmentation project with softseg
MIT License
4 stars 3 forks source link

Slow repo cloning #86

Closed valosekj closed 11 months ago

valosekj commented 11 months ago

Cloning of this repo is very slow (it took me almost 10 minutes)

valosek@macbook-pro:~/code$ git clone git@github.com:sct-pipeline/contrast-agnostic-softseg-spinalcord.git
Cloning into 'contrast-agnostic-softseg-spinalcord'...
remote: Enumerating objects: 6846, done.
remote: Counting objects: 100% (1465/1465), done.
remote: Compressing objects: 100% (478/478), done.
Receiving objects:  16% (1156/6846), 39.69 MiB | 189.00 KiB/s

Maybe due to a large number of objects (6846)?

jcohenadad commented 11 months ago

Thank you for reporting this Jan, this is indeed concerning. At first I thought the issue was the "189.00 KiB/s", but I have a 60MB/s connection and my clone is showing approximately 300-500kB/s, and is also taking ~10min. Maybe @mguaypaq @namgo @kousu have some insights

mguaypaq commented 11 months ago

Having (slowly) cloned the repository just now, I believe it's because of the large total size (237MB), rather than the number of objects. For example, the SCT repository takes a little bit less time to clone, but has >60k objects.

It looks like most of that large size comes from binary files that were added and later deleted in the branch lfb/monai_aggregate. Specifically, the commit 06eafedbe41c136c5f6794d7184cae07115d8542 contains a folder wandb which is 332MB.

Since @louisfb01 is no longer at the lab, can we delete this branch? That should make the repository small again.

kousu commented 11 months ago

Good investigation @mguaypaq.

We could also rebase his branch to add a '.gitignore' + delete wandb/ to https://github.com/sct-pipeline/contrast-agnostic-softseg-spinalcord/commit/06eafedbe41c136c5f6794d7184cae07115d8542 without losing the rest of the branch. But yeah, it seems like probably we don't really need it...

jcohenadad commented 11 months ago

Since @louisfb01 is no longer at the lab, can we delete this branch? That should make the repository small again.

yes, definitely, thank you @mguaypaq and @kousu for your help

mguaypaq commented 11 months ago

Done. And now that the lfb/monai_aggregate branch has been deleted, it takes less than 1 second to clone the repository:

remote: Enumerating objects: 1896, done.
remote: Counting objects: 100% (1290/1290), done.
remote: Compressing objects: 100% (451/451), done.
remote: Total 1896 (delta 935), reused 1102 (delta 839), pack-reused 606
Receiving objects: 100% (1896/1896), 430.73 KiB | 5.13 MiB/s, done.
Resolving deltas: 100% (1211/1211), done.

real    0m0.746s
user    0m0.101s
sys 0m0.033s
valosekj commented 11 months ago

Thank you for the quick fix!