echemdata / galvani

Read proprietary file formats from electrochemical test stations
GNU General Public License v3.0
47 stars 30 forks source link

CI is using up too much git-lfs bandwidth #100

Open chatcannon opened 9 months ago

chatcannon commented 9 months ago

GitHub puts a quota on git-lfs bandwidth usage, as well as storage space. We are nowhere near filling the storage limit but because every CI run pulls down all the LFS data once for every version of python tested, it eats up the free quota very quickly.

We wanted to let you know that you’ve used 80% of your data plan for Git LFS on the organization echemdata. No immediate action is necessary, but you might want to consider purchasing additional data packs to cover your bandwidth and storage usage:

https://github.com/organizations/echemdata/billing/data/upgrade

Current usage as of 03 Feb 2024 04:21PM UTC:

Bandwidth: 0.8 GB / 1 GB (80%) Storage: 0.01 GB / 1 GB (1%)

chatcannon commented 9 months ago

@ml-evs I don't know much about the new github actions CI setup - can you look into this? There ought to be a way to cache data between runs.

ml-evs commented 9 months ago

I've just opened #101 which hopefully fixes this, making use of https://github.com/nschloe/action-cached-lfs-checkout until GitHub adds native support for this (or ideally stops charging for internal network bandwidth). I think on the free "academic team" plan on GitHub the LFS quota is much more generous. Depending on how you view things, the echemdata organisation might be upgradable to this tier as long as someone in the organisation retains an academic affiliation (and as long as the org remains to be used for stuff like galvani).

ml-evs commented 8 months ago

101 doesn't seem to be having much effect, I'm not sure if this is because the bandwidth for the month is already exceeded. Unfortunately this is a product of not having an up to date release on PyPI, so anyone who wants to install the dev version is either failing and using up git lfs bandwidth, or is having to disable LFS.

chatcannon commented 8 months ago

I still got github actions errors yesterday even though it is the beginning of a new month.

I think this might be the last straw that finally pushes me over the edge to migrate to https://sourcehut.org/ - I would certainly rather pay them than Microsoft.