FYI: Project over Github's git-lfs quota

[pfalcon]

To reproduce:

$ git clone --depth 1 https://github.com/ARM-software/CMSIS_5.git -b 5.7.0
Cloning into 'CMSIS_5'...
remote: Enumerating objects: 8352, done.
remote: Counting objects: 100% (8352/8352), done.
remote: Compressing objects: 100% (5661/5661), done.
remote: Total 8352 (delta 3731), reused 5490 (delta 2471), pack-reused 0
Receiving objects: 100% (8352/8352), 39.27 MiB | 2.34 MiB/s, done.
Resolving deltas: 100% (3731/3731), done.
Note: switching to 'a65b7c9a3e6502127fdb80eb288d8cbdf251a6f4'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

Updating files: 100% (9817/9817), done.
Downloading CMSIS/DSP/Lib/ARM/arm_ARMv8MBLl_math.lib (2.8 MB)
Error downloading object: CMSIS/DSP/Lib/ARM/arm_ARMv8MBLl_math.lib (496f8cd): Smudge error: Error downloading CMSIS/DSP/Lib/ARM/arm_ARMv8MBLl_math.lib (496f8cdab57174e66eb6cfb365cba46298b86440ddaf1660176bcac0566aebb6): batch response: This repository is over its data quota. Account responsible for LFS bandwidth should purchase more data packs to restore access.

Errors logged to /mnt/hdd/projects-3rdparty/Embedded/TF-M/test/a/CMSIS_5/.git/lfs/logs/20210514T152345.108587133.log
Use `git lfs logs last` to view the log.
error: external filter 'git-lfs filter-process' failed
fatal: CMSIS/DSP/Lib/ARM/arm_ARMv8MBLl_math.lib: smudge filter lfs failed
warning: Clone succeeded, but checkout failed.
You can inspect what was checked out with 'git status'
and retry with 'git restore --source=HEAD :/'

This actually happened in our (Linaro's) CI builds for TF-M, and the command came from some TF-M build manual (well, it used 5.5.0). On our side, we switch to download a pack file, but I decided to submit this ticket to keep you in loop on such matters (I don't see such a recent issue around).

[JonatanAntoni]

Hi @pfalcon,

thanks for letting me know. The problem here is that pulling the LFS files all over again is overusing our bandwidth quota. Unfortunately, this does not only apply to your CI at Linaro and not only when cloning the upstream repo. It affects all forks and third party systems in the same way.

Last time we agreed on keeping the LFS data in place so that accessing older release tags are not breaking. But as we've now seen a massive increase of LFS bandwidth usage we might not be able to sustain this any further.

May I ask you to check with all TF-M users to not clone CMSIS all over again, please. CI systems should keep the files in a local cache such as using a local reference repository. If your builds do not require the pre-built libraries stored on Git LFS, please disable pulling these files.

If we cannot get the bandwidth consumption down we may be forced to purge LFS data and accept breaking release tags.

Thanks, Jonatan

[pfalcon]

Unfortunately, this does not only apply to your CI at Linaro and not only when cloning the upstream repo. It affects all forks and third party systems in the same way.

Sure. Again, I didn't see a recent ticket on that issue (and we never saw such an issue before, even though we ran TF-M ci job for at least several months).

May I ask you to check with all TF-M users to not clone CMSIS all over again, please.

Well, in this case, we're downstream users of the upstream TF-M project, and we just followed the instruction the TF-M project had (at that time). As I mentioned, first thing I did is investigated alternatives to git-lfs and switched to downloading a .pack file. It also seems that our build setup is a bit dated and newer TF-M versions avoid making git-lfs pull from this repository (but not sure if it just switched to another). I'll double-check and let you know.

CI systems should keep the files in a local cache such as using a local reference repository. If your builds do not require the pre-built libraries stored on Git LFS, please disable pulling these files.

I personally fully agree, and going to pass this word to my management, and work on cleanups/optimizations. But otherwise, it's mostly a matter of following the upstream build instructions as is (if we start to change them, there's immediately a question if we actually build an upstream project or some "fork" of it; again, I agree that reasonable optimizations should be made).

Thanks for the response. I'm not closing this ticket right away, so other users which may be affected could easily find it and make corrections on their side. But of course, feel free to close it as you see fit.

[Anton-TF]

Let me note that upstream TF-M addressed this issue (LFS quota) about a year ago (May 2020) by holding a small portion of CMSIS in tf-m-test repo (https://git.trustedfirmware.org/TF-M/tf-m-tests.git/tree/CMSIS) for testing on NS. TF-M is independent of this repo (and LFS) since v1.1 so suggest updating to the latest TF-M version.

[arurke]

I believe the repository has met the limit once more. I am seeing "This repository is over its data quota. Account responsible for LFS bandwidth should purchase more data packs". And it seems I cannot even pull the repository to make a local fork so as to not need to pull from github in the future. Is there anything you can do on your side?

[JonatanAntoni]

Hi @arurke,

We can't do anything about it. It was a misconception at our end to use LFS at all. We "solved" this by splitting CMSIS components into separate new repos. If possible, please update to CMSIS 6 et al. Or, if you depend on CMSIS 5, make use of the release bundle instead of cloning the repo.

[arurke]

Thanks for the quick reply @JonatanAntoni . Turns out we did not need the LFS files, so I simply skipped it by setting GIT_LFS_SKIP_SMUDGE: "1". I will make a local fork when your quota is reset. But for sure I will check out CMSIS 6 - I see you have a migration guide as well :+1:

[JonatanAntoni]

Thanks for the quick reply @JonatanAntoni . Turns out we did not need the LFS files, so I simply skipped it by setting GIT_LFS_SKIP_SMUDGE: "1". I will make a local fork when your quota is reset. But for sure I will check out CMSIS 6 - I see you have a migration guide as well 👍

Please be aware that a fork will not duplicate LFS files. This fact is the caveat that is hitting us: All the forks of CMSIS_5 repo consume our LFS bandwidth.