nice-registry / all-the-package-repos

🌍 Normalized repository URLs for every package in the npm registry. Updated daily.
78 stars 9 forks source link

Error cloning the repository due to "lack of data packs" (LFS) #42

Closed bbossola closed 2 years ago

bbossola commented 2 years ago

Hi guys, it seems that's is now problematic to clone the repository, as an error from GitHub is reported:

git clone --depth=1 https://github.com/nice-registry/all-the-package-repos
Cloning into 'all-the-package-repos'...
remote: Enumerating objects: 29, done.
remote: Counting objects: 100% (29/29), done.
remote: Compressing objects: 100% (24/24), done.
remote: Total 29 (delta 0), reused 17 (delta 0), pack-reused 0
Unpacking objects: 100% (29/29), done.
Downloading data/packages.json (108 MB)
Error downloading object: data/packages.json (2970784): Smudge error: Error downloading data/packages.json (2970784795da51409d84243b45dfe0edbb22bd86d283e99b604d1ebd38a89829): batch response: This repository is over its data quota. Account responsible for LFS bandwidth should purchase more data packs to restore access.

Errors logged to '/tmp/all-the-package-repos/.git/lfs/logs/20220513T170018.111632974.log'.
Use `git lfs logs last` to view the log.
error: external filter 'git-lfs filter-process' failed
fatal: data/packages.json: smudge filter lfs failed
warning: Clone succeeded, but checkout failed.
You can inspect what was checked out with 'git status'
and retry the checkout with 'git checkout -f HEAD'

I can confirm that also using the API to retrieve that specific files, it fails with the same error:

curl -X POST \
-H "Accept: application/vnd.git-lfs+json" \
-H "Content-type: application/json" \
-d '{"operation": "download", "transfer": ["basic"], "objects": [{"oid": "2970784795da51409d84243b45dfe0edbb22bd86d283e99b604d1ebd38a89829", "size": 107728556}]}' \
https://github.com/nice-registry/all-the-package-repos.git/info/lfs/objects/batch

(I followed a simple Gist available at https://gist.github.com/fkraeutli/66fa741d9a8c2a6a238a01d17ed0edc5 )

Any advice?

MartinKolarik commented 2 years ago

Hi, yes, we're aware of this and currently discussing how to resolve this.

jimaek commented 2 years ago

@zeke I emailed you about this a few days ago, please check when you can

MartinKolarik commented 2 years ago

@bbossola in the meantime you can still get the files from npm.

bbossola commented 2 years ago

Thanks. Atm I got a slightly old version from the commit before the move to LFS: https://raw.githubusercontent.com/nice-registry/all-the-package-repos/b30c5b80fa1331bf09eda98b0b40bdb63f5e7bb6/data/packages.json

How would I get the files from npm?

MartinKolarik commented 2 years ago

This has been resolved.

zeke commented 2 years ago

Thank you for sorting it out! Just so we're clear for posterity, @jimaek pitched in and upgraded the @nice-registry org to include one "data pack" for more Git LFS storage and bandwidth.

Screen Shot 2022-05-16 at 12 07 32 PM

Thanks so much for taking care of it! 🙏🏼