ropensci / piggyback

:package: for using large(r) data files on GitHub
https://docs.ropensci.org/piggyback
GNU General Public License v3.0
182 stars 26 forks source link

Maximum overall non-code release size #98

Closed MiguelRodo closed 1 year ago

MiguelRodo commented 1 year ago

Hi

Thank you for a fantastic package.

In starting to use the package, I want to get an idea of how large we can make our non-code releases without upsetting GitHub. I know that GitHub has a 2GB/file limit for releases, but the overall release size has no hard limit. So, how big have people have made non-code GitHub releases before (with or without piggyback)? Or what might one expect GitHub to reasonably be okay with?

Note that I would intend to overwrite old non-code releases and archive locally or with another provider. That way, only the latest version of the non-code items would be shared via GitHub rather than all their variants. Also worth noting that the items to be uploaded are either to be processed by code (e.g. raw data) or produced by it (e.g. HTML docs), so whilst not source or compiled code themselves they are closely linked to code.

Thanks!

Note: I would ask this on Stackoverflow, since it's neither a bug nor a feature request, but there isn't a piggyback tag and the term "piggyback" is used often in other contexts.

cboettig commented 1 year ago

Good question, but no idea what the answer is! My own take is that it would be up to GitHub to implement their own limits, so just give it a try? (and let us know how it goes!)

But also note that GitHub can obviously change those limits at any time, (including the 2 GB/file limit), so storage in this fashion should never be considered archival! Keep in mind Zenodo has a 50 GB limit per submission.

MiguelRodo commented 1 year ago

Sure, will post back if I ever get told off! Thanks for the response, and the tip re Zenodo.