search

poldracklab / pydeface

defacing utility for MRI images
MIT License
110 stars 42 forks source link

Clone depth #26

Closed muschellij2 closed 4 years ago

muschellij2 commented 5 years ago

Just not needing full history if installing.

vsoch commented 5 years ago

Overall Idea

This is pretty neat (I didn't know about shallow clones)! If the user is just installing and doesn't need to develop, or get history for any reason, this should be okay. It's only problematic in the case of fork and clone for purposes of developing pydeface. And it looks like for some earlier versions of git they were limited in terms of push and pull.

The only danger would be the case where the user pulls (shallow), forgets that it's shallow, and then attempts a pull request (and I'm not sure about the behavior that would result). I would add a note, something like:

To install:

git clone https://github.com/poldracklab/pydeface.git
git clone --depth 1 https://github.com/poldracklab/pydeface.git
cd pydeface
python setup.py install

important if you plan to further develop the library, you would be best to remove the --depth 1 variable to preserve the repository for this purpose. If you are likely to forget about the shallow clone, you could remove the downloaded repository:

cd ..
rm -rf pydeface

and re-obtain it when an upgrade is needed.

Is it an improvement?

I can confirm it only grabs one level of commit:

$ cd pydeface-shallow/
$ git log
commit cc38ca17c7b5a975fc908748fa1d52c1ebbc1d35
Author: Shotgunosine <adenosine@gmail.com>
Date:   Tue Feb 5 10:43:50 2019 -0500

    FIX circleci config

If you are doing this for the time saved cloning, it's not worth it:

$ time git clone --depth 1 https://github.com/poldracklab/pydeface.git pydeface-shallow
...
real    0m2.274s
user    0m1.018s
sys 0m0.206s
$ time git clone https://github.com/poldracklab/pydeface.git
...
real    0m1.663s
user    0m0.483s
sys 0m0.246s

Are you able to see that there is significant difference in the size of the repos? I might be doing the wrong command, but I don't see it.

pydeface-shallow$ du -hs .[!.]* *
8.0K    .circleci
20M .git
4.0K    .gitignore
8.0K    Dockerfile
4.0K    LICENSE
4.0K    MANIFEST.in
8.3M    pydeface
4.0K    README.md
4.0K    setup.py
12M tests
pydeface$ du -hs .[!.]* *
8.0K    .circleci
20M .git
4.0K    .gitignore
8.0K    Dockerfile
4.0K    LICENSE
4.0K    MANIFEST.in
8.3M    pydeface
4.0K    README.md
4.0K    setup.py
12M tests

So - I like the spirit of this idea, but I'm not convinced it's necessary. @muschellij2 would you care to show what you found to be an improvement?

muschellij2 commented 5 years ago

I got much smaller .git size:

[jmuschel@compute-106 ~]$ time git clone --depth 1 https://github.com/poldracklab/pydeface.git pydeface-shallow
Initialized empty Git repository in /users/jmuschel/pydeface-shallow/.git/
remote: Enumerating objects: 44, done.
remote: Counting objects: 100% (44/44), done.
remote: Compressing objects: 100% (36/36), done.
remote: Total 44 (delta 8), reused 29 (delta 2), pack-reused 0
Unpacking objects: 100% (44/44), done.

real    0m3.543s
user    0m2.254s
sys 0m0.264s
[jmuschel@compute-106 ~]$ cd pydeface-shallow
[jmuschel@compute-106 pydeface-shallow]$ du -hs .[!.]* *
1.0K    .circleci
431K    .git
512 .gitignore
512 Dockerfile
512 LICENSE
512 MANIFEST.in
3.5K    pydeface
512 README.md
512 setup.py
2.5K    tests
[jmuschel@compute-106 pydeface-shallow]$ cd ..
[jmuschel@compute-106 ~]$ 
[jmuschel@compute-106 ~]$ time git clone https://github.com/poldracklab/pydeface.git 

Initialized empty Git repository in /users/jmuschel/pydeface/.git/
remote: Enumerating objects: 50, done.
remote: Counting objects: 100% (50/50), done.
remote: Compressing objects: 100% (20/20), done.
remote: Total 307 (delta 21), reused 48 (delta 19), pack-reused 257
Receiving objects: 100% (307/307), 19.44 MiB, done.
Resolving deltas: 100% (161/161), done.

real    0m1.564s
user    0m0.468s
sys 0m0.256s
[jmuschel@compute-106 ~]$ cd pydeface
[jmuschel@compute-106 pydeface]$ du -hs .[!.]* *
1.0K    .circleci
6.4M    .git
512 .gitignore
512 Dockerfile
512 LICENSE
512 MANIFEST.in
3.5K    pydeface
512 README.md
512 setup.py
2.5K    tests
vsoch commented 5 years ago

That's really weird - let me do it again. What is your version of git?

vsoch commented 5 years ago

It's not... reproducible!! :scream:

Oh git, I am so disappointed. Check this out, it's 20M for me both ways, again :/

vanessa@vanessa-ThinkPad-T460s:/tmp$ time git clone --depth 1 https://github.com/poldracklab/pydeface.git pydeface-shallow
Cloning into 'pydeface-shallow'...
remote: Enumerating objects: 22, done.
remote: Counting objects: 100% (22/22), done.
remote: Compressing objects: 100% (19/19), done.
remote: Total 22 (delta 0), reused 16 (delta 0), pack-reused 0
Unpacking objects: 100% (22/22), done.
Checking connectivity... done.

real    0m3.965s
user    0m1.083s
sys 0m0.223s
vanessa@vanessa-ThinkPad-T460s:/tmp$ cd pydeface-shallow/
vanessa@vanessa-ThinkPad-T460s:/tmp/pydeface-shallow$ du -hs .[!.]* *
8.0K    .circleci
20M .git
4.0K    .gitignore
8.0K    Dockerfile
4.0K    LICENSE
4.0K    MANIFEST.in
8.3M    pydeface
4.0K    README.md
4.0K    setup.py
12M tests
vanessa@vanessa-ThinkPad-T460s:/tmp/pydeface-shallow$ cd ..
vanessa@vanessa-ThinkPad-T460s:/tmp$  time git clone https://github.com/poldracklab/pydeface.git 
Cloning into 'pydeface'...
remote: Enumerating objects: 50, done.
remote: Counting objects: 100% (50/50), done.
remote: Compressing objects: 100% (20/20), done.
remote: Total 307 (delta 21), reused 48 (delta 19), pack-reused 257
Receiving objects: 100% (307/307), 19.44 MiB | 9.81 MiB/s, done.
Resolving deltas: 100% (161/161), done.
Checking connectivity... done.

real    0m3.032s
user    0m0.568s
sys 0m0.313s
vanessa@vanessa-ThinkPad-T460s:/tmp$ cd pydeface
vanessa@vanessa-ThinkPad-T460s:/tmp/pydeface$ du -hs .[!.]* *
8.0K    .circleci
20M .git
4.0K    .gitignore
8.0K    Dockerfile
4.0K    LICENSE
4.0K    MANIFEST.in
8.3M    pydeface
4.0K    README.md
4.0K    setup.py
12M tests
vanessa@vanessa-ThinkPad-T460s:/tmp/pydeface$ git --version
git version 2.7.4
muschellij2 commented 5 years ago

git version 2.16.2 John

On Wed, Apr 10, 2019 at 1:31 PM Vanessasaurus notifications@github.com wrote:

It's not... reproducible!! 😱

Oh git, I am so disappointed. Check this out, it's 20M for me both ways, again :/

vanessa@vanessa-ThinkPad-T460s:/tmp$ time git clone --depth 1 https://github.com/poldracklab/pydeface.git pydeface-shallow

Cloning into 'pydeface-shallow'...

remote: Enumerating objects: 22, done.

remote: Counting objects: 100% (22/22), done.

remote: Compressing objects: 100% (19/19), done.

remote: Total 22 (delta 0), reused 16 (delta 0), pack-reused 0

Unpacking objects: 100% (22/22), done.

Checking connectivity... done.

real 0m3.965s

user 0m1.083s

sys 0m0.223s

vanessa@vanessa-ThinkPad-T460s:/tmp$ cd pydeface-shallow/

vanessa@vanessa-ThinkPad-T460s:/tmp/pydeface-shallow$ du -hs .[!.]

8.0K .circleci

20M .git

4.0K .gitignore

8.0K Dockerfile

4.0K LICENSE

4.0K MANIFEST.in

8.3M pydeface

4.0K README.md

4.0K setup.py

12M tests

vanessa@vanessa-ThinkPad-T460s:/tmp/pydeface-shallow$ cd ..

vanessa@vanessa-ThinkPad-T460s:/tmp$ time git clone https://github.com/poldracklab/pydeface.git

Cloning into 'pydeface'...

remote: Enumerating objects: 50, done.

remote: Counting objects: 100% (50/50), done.

remote: Compressing objects: 100% (20/20), done.

remote: Total 307 (delta 21), reused 48 (delta 19), pack-reused 257

Receiving objects: 100% (307/307), 19.44 MiB | 9.81 MiB/s, done.

Resolving deltas: 100% (161/161), done.

Checking connectivity... done.

real 0m3.032s

user 0m0.568s

sys 0m0.313s

vanessa@vanessa-ThinkPad-T460s:/tmp$ cd pydeface

vanessa@vanessa-ThinkPad-T460s:/tmp/pydeface$ du -hs .[!.]

8.0K .circleci

20M .git

4.0K .gitignore

8.0K Dockerfile

4.0K LICENSE

4.0K MANIFEST.in

8.3M pydeface

4.0K README.md

4.0K setup.py

12M tests

vanessa@vanessa-ThinkPad-T460s:/tmp/pydeface$ git --version

git version 2.7.4

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/poldracklab/pydeface/pull/26#issuecomment-481786665, or mute the thread https://github.com/notifications/unsubscribe-auth/ABBnrlcaL3ZaQBqDM8DvVoDD1rSNeoByks5vfh_TgaJpZM4cmB1f .

vsoch commented 5 years ago

Mine is a bit old then. Let me update, and test again.

vsoch commented 5 years ago

Just ran it again with

$ git --version
git version 2.21.0

I don't see any changes. I'm not sure how to proceed, maybe someone else can test.

effigies commented 5 years ago

git 2.19 on OSX, no effect of --depth 1.

vsoch commented 5 years ago

Thanks for testing @effigies.

ofgulban commented 4 years ago

I think this is not an issue anymore since pypi version is updated and pip install pydeface is added to the readme.