search

mozilla / webcompat-ml-docs

Documentation about ML in webcompat
10 stars 0 forks source link

git clone failure - Clone succeeded, but checkout failed. #1

Open karlcow opened 4 years ago

karlcow commented 4 years ago

@johngian @miketaylr Reading and following the docs https://webcompat-ml.readthedocs.io/en/latest/installation.html

% git clone https://github.com/mozilla/webcompat-ml
Cloning into 'webcompat-ml'...
remote: Enumerating objects: 301, done.
remote: Counting objects: 100% (301/301), done.
remote: Compressing objects: 100% (171/171), done.
remote: Total 301 (delta 122), reused 272 (delta 93), pack-reused 0
Receiving objects: 100% (301/301), 242.01 KiB | 231.00 KiB/s, done.
Resolving deltas: 100% (122/122), done.
git-lfs filter-process: git-lfs: command not found
fatal: the remote end hung up unexpectedly
warning: Clone succeeded, but checkout failed.
You can inspect what was checked out with 'git status'
and retry with 'git restore --source=HEAD :/'

ok let's see what that means

You can inspect what was checked out with 'git status'

cd webcompat-ml
git status

it gives me:

On branch master
Your branch is up to date with 'origin/master'.

Changes to be committed:
  (use "git restore --staged <file>..." to unstage)
    deleted:    .gitattributes
    deleted:    .gitignore
    deleted:    Dockerfile
    deleted:    LICENSE
    deleted:    README.md
    deleted:    datasets/all-issues-events.csv
    deleted:    datasets/needsdiagnosis-balanced-original-titles.csv
    deleted:    datasets/needsdiagnosis-full-original-titles.csv
    deleted:    docs/architecture.md
    deleted:    docs/pipeline.md
    deleted:    setup.py
    deleted:    src/webcompat_ml/__init__.py
    deleted:    src/webcompat_ml/models/__init__.py
    deleted:    src/webcompat_ml/models/needsdiagnosis/__init__.py
    deleted:    src/webcompat_ml/models/needsdiagnosis/cli.py
    deleted:    src/webcompat_ml/models/needsdiagnosis/model.py
    deleted:    src/webcompat_ml/utils/__init__.py
    deleted:    src/webcompat_ml/utils/data.py

Untracked files:
  (use "git add <file>..." to include in what will be committed)
    .gitattributes
    .gitignore
    Dockerfile
    LICENSE
    README.md

and

git restore --source=HEAD :/

gives:

git-lfs filter-process: git-lfs: command not found
fatal: the remote end hung up unexpectedly
karlcow commented 4 years ago

I guess it requires https://git-lfs.github.com/

karlcow commented 4 years ago

So for me what I did:

then 

~/Downloads/git-lfs-darwin-amd64-v2.10.0 % ./install.sh

ah then we run into a loop of not accepted because of security settings on the mac. Though it seems to accept it at the same time.

~/Downloads/git-lfs-darwin-amd64-v2.10.0 % ls -al /usr/local/bin/git-lfs 
-rwxr-xr-x@ 1 root  admin  13148744 27 fév 09:22 /usr/local/bin/git-lfs

ok you need to manually activate it. 

~/Downloads/git-lfs-darwin-amd64-v2.10.0 % git lfs install
Git LFS initialized.

There will be a pop up and you can select open.

karlcow commented 4 years ago

Let's go back to the webcompat-ml directory

and let's type the initial command.

~/code/webcompat-ml % git restore --source=HEAD :/
Filtering content: 100% (3/3), 764.84 MiB | 9.12 MiB/s, done.
~/code/webcompat-ml % git status
On branch master
Your branch is up to date with 'origin/master'.

Changes to be committed:
  (use "git restore --staged <file>..." to unstage)
    deleted:    .gitattributes
    deleted:    .gitignore
    deleted:    Dockerfile
    deleted:    LICENSE
    deleted:    README.md
    deleted:    datasets/all-issues-events.csv
    deleted:    datasets/needsdiagnosis-balanced-original-titles.csv
    deleted:    datasets/needsdiagnosis-full-original-titles.csv
    deleted:    docs/architecture.md
    deleted:    docs/pipeline.md
    deleted:    setup.py
    deleted:    src/webcompat_ml/__init__.py
    deleted:    src/webcompat_ml/models/__init__.py
    deleted:    src/webcompat_ml/models/needsdiagnosis/__init__.py
    deleted:    src/webcompat_ml/models/needsdiagnosis/cli.py
    deleted:    src/webcompat_ml/models/needsdiagnosis/model.py
    deleted:    src/webcompat_ml/utils/__init__.py
    deleted:    src/webcompat_ml/utils/data.py

Untracked files:
  (use "git add <file>..." to include in what will be committed)
    .gitattributes
    .gitignore
    Dockerfile
    LICENSE
    README.md
    datasets/
    docs/
    setup.py
    src/

@johngian do you know why it is in this state?

danielwangksu commented 3 years ago

Having the same issue on some other repository as well any idea? Thanks!

ksy36 commented 3 years ago

So I've tried to clone https://github.com/mozilla/webcompat-ml an got the same error:

git-lfs filter-process: git-lfs: command not found
fatal: the remote end hung up unexpectedly
warning: Clone succeeded, but checkout failed.

Installed git-lfs:

brew install git-lfs

And another error:

ksenia$ git clone https://github.com/mozilla/webcompat-ml
Cloning into 'webcompat-ml'...
remote: Enumerating objects: 301, done.
remote: Total 301 (delta 0), reused 0 (delta 0), pack-reused 301
Receiving objects: 100% (301/301), 242.10 KiB | 2.58 MiB/s, done.
Resolving deltas: 100% (122/122), done.
Downloading datasets/all-issues-events.csv (692 MB)
Error downloading object: datasets/all-issues-events.csv (2d50aec): Smudge error: Error downloading datasets/all-issues-events.csv 
batch response: This repository is over its data quota. Account responsible for LFS bandwidth should purchase more data packs to restore access.

One of the csv files is very big (692 MB). But we have a bandwidth of 1GB per month (so I wonder why I reached the limit here)

From https://docs.github.com/en/github/managing-large-files/about-storage-and-bandwidth-usage:

Every account using Git Large File Storage receives 1 GB of free storage and 1 GB a month of free bandwidth. If the bandwidth and storage quotas are not enough, you can choose to purchase an additional quota for Git LFS.

For example: If you push a 500 MB file to Git LFS, you'll use 500 MB of your allotted storage and none of your bandwidth. If you make a 1 byte change and push the file again, you'll use another 500 MB of storage and no bandwidth, bringing your total usage for these two pushes to 1 GB of storage and zero bandwidth. If you download a 500 MB file that's tracked with LFS, you'll use 500 MB of the repository owner's allotted bandwidth. If a collaborator pushes a change to the file and you pull the new version to your local repository, you'll use another 500 MB of bandwidth, bringing the total usage for these two downloads to 1 GB of bandwidth.

Then tried the following to ignore large files:

GIT_LFS_SKIP_SMUDGE=1 git clone https://github.com/mozilla/webcompat-ml

And finally was able to checkout, but the csv files are only downloaded as pointer files. https://docs.github.com/en/github/managing-large-files/about-git-large-file-storage#pointer-file-format

Git LFS handles large files by storing references to the file in the repository, but not the actual file itself. To work around Git's architecture, Git LFS creates a pointer file which acts as a reference to the actual file (which is stored somewhere else). GitHub manages this pointer file in your repository. When you clone the repository down, GitHub uses the pointer file as a map to go and find the large file for you.

MatrixMike commented 2 years ago

It seems to happen when I clone on Windows 8 or 10. The same repo clones fine on Linux - various versions. The repo in question for me is https://github.com/8l/bcpl . Maybe the repo was created with another version and is 'old' {6 years} and needs some manual intervention.