liberapay / git-lfs-fetch.py

Lightweight Git Large File Storage fetcher written in python
31 stars 13 forks source link

project goal description is missing #2

Closed volviq closed 7 years ago

volviq commented 7 years ago

The Readme only shows how to launch the program in one way. It does not explain, what purposes the application is written for and what are its limitations. It would be good to extend the Readme.md by adding that information. Otherwise one could think this can be used as a full git client replacement.

Changaco commented 7 years ago

The README seems clear enough to me, it says "A lightweight Git Large File Storage fetcher written in python", and it shows the output of python -m git_lfs -h. However since you've taken the time to open this issue I guess it's not clear to everybody. Suggestions or pull requests to improve clarity are welcome.

volviq commented 7 years ago

Actually I didn't fully understand what the "git lfs fetcher" really does for me. So adding more description would at least help me to find out if the tool does what I need, or something else. I assumed, I can use a normal git client for cloning and then get those lfs based files of the repository witthout githubs go based git lfs application - e.g. for platforms which do not have go lang support.
The tool did something, but in the end I only got some files in a .git folder - nothing checked out.

Changaco commented 7 years ago

I assumed, I can use a normal git client for cloning and then get those lfs based files of the repository witthout githubs go based git lfs application - e.g. for platforms which do not have go lang support.

That is indeed what this project is for.

The tool did something, but in the end I only got some files in a .git folder - nothing checked out.

What command(s) did you run exactly? Did you see any error message?

volviq commented 7 years ago

Readme only show me "python -m git_lfs -h" as example invocation. And that gives me:

usage: __main__.py [-h] [-v] [git_repo] [checkout_dir]

positional arguments:
  git_repo       if it's bare you need to provide a checkout_dir
  checkout_dir

optional arguments:
  -h, --help     show this help message and exit
  -v, --verbose

I would miss a short step by step instruction how to make use of the tool.

E.g. I tried:

git  clone [repo url]
cd [repo  dir]
python -m git_lfs 

and then checked, if I have the LFS files downloaded and if it did overwrite the 133 bytes sized LFS references. And it did not. --> nothing seems to happen.

git  clone [repo url]
cd [repo  dir]
python -m git_lfs [repo url]
#gives the reply:
Can't checkout into a bare repo, please provide a valid checkout_dir

I also tried to execute it like: python -m git_lfs [rep url] [repo dir]

and it downloaded something, but again - no effect on the checked out master. It seems I am missing a step or I have a wrong understanding of the intended use of the tool.

Changaco commented 7 years ago

The first thing you tried is correct. Can you do it again with -vv (python -m git_lfs -vv) and paste the output so we can see what's going on? Maybe the module isn't detecting the LFS files for some reason. Can you also paste the output of git --version?

volviq commented 7 years ago
python -m git_lfs -vv
Nothing to fetch.

git version 1.9.1

Changaco commented 7 years ago

Okay, so it's not seeing the LFS files. The server I'm using this project on has git 1.8.2.1, so your version isn't older.

What's in your repo's .gitattributes file? Does git ls-files | git check-attr --cached --stdin diff | grep lfs output the list of all files stored with LFS?

volviq commented 7 years ago
cat .gitattributes 

#
# Git LFS (see https://git-lfs.github.com/)
#
*.bz2 filter=lfs -text
*.exe filter=lfs -text
*.gz filter=lfs -text
*.jar filter=lfs -text
*.pdf filter=lfs -text
*.tar filter=lfs -text
*.zip filter=lfs -text
 git ls-files | git check-attr --cached --stdin diff | grep lfs

gives me empty results. I have intentionally uninstalled the "normal" git lfs client to see if this tool is a full replacement. before that I had that repo cloned successfully with git lfs and I got those files (I can recheck though). I'm using bitbucked, and clicking on .gz files indeed marks them as "stored in lfs"

E.g. I get:

git ls-files | git check-attr --cached --stdin diff | grep gz
pysrc/archives/Pyrex-0.9.6.2.tar.gz: diff: unspecified
Changaco commented 7 years ago

Well, we've found the problem, your .gitattributes only sets filter=lfs, while mine also has diff=lfs merge=lfs. The git_lfs module only looks at the diff attribute. I'll modify it to look at filter too.

volviq commented 7 years ago

thanks! How should the .gitattributes then look like? I changed my lines into: *.bz2 filter=lfs diff=lfs merge=lfs -text but the command: git ls-files | git check-attr --cached --stdin diff | grep lfs still doesn't show me anything

Changaco commented 7 years ago

If you haven't committed the change you need to drop the --cached flag in the command.

volviq commented 7 years ago

I commited it, the ls-files now gives me the lfs files, but now I get:

python -m git_lfs -vv

Traceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/usr/local/lib/python2.7/dist-packages/git_lfs/__main__.py", line 14, in <module>
    fetch(args.git_repo, args.checkout_dir, args.verbose)
  File "/usr/local/lib/python2.7/dist-packages/git_lfs/__init__.py", line 155, in fetch
    lfs_url = get_lfs_endpoint_url(git_repo, checkout_dir)
  File "/usr/local/lib/python2.7/dist-packages/git_lfs/__init__.py", line 47, in get_lfs_endpoint_url
    url = urlunsplit('https', url.hostname, url.path)
TypeError: urlunsplit() takes exactly 1 argument (3 given)
Changaco commented 7 years ago

That definitely looks like a stupid bug.

volviq commented 7 years ago

Output of: git remote -v looks similar to: ssh://git@git.url.loc:3333/~username/repo_git_lfs.git

Changaco commented 7 years ago

Everything should be fixed now. I've tagged a new version and uploaded it to PyPI.