search

Kreyren / kreyren

Personal tracking for issues that i need to resolve to be used as a reference for someone else and/or for peer-review of the solution
GNU General Public License v3.0
3 stars 0 forks source link

New infrastructure for NixOS #110

Open Kreyren opened 3 months ago

Kreyren commented 3 months ago

TBD

Storage requirements: 165 TB/year expected linear growth of up to 15% Average Network Saturation: 10gbps

Budget: TBD

  1. https://discourse.nixos.org/t/the-nixos-foundations-call-to-action-s3-costs-require-community-support/28672
Kreyren commented 3 months ago

Referencing: https://github.com/NixOS/nixpkgs/pull/212930#issuecomment-1436011059

Hey @MatthewCroughan

How far you got with that config.services.seed-nixpkgs.enable = true idea? I would like to explore the option of having a seedboxes that distribute the signed nars and derivations from hydra as an alternative to the current solution of cache.nixos.org

MatthewCroughan commented 3 months ago

Yeah it would be nice, but haven't been motivated to implement it yet. It's the kind of thing I'd expect a grant to fund.

Kreyren commented 3 months ago

It's the kind of thing I'd expect a grant to fund. -- @MatthewCroughan (https://github.com/Kreyren/kreyren/issues/110#issuecomment-2291300223)

From conversation in NixOS channel it felt like the limitation is the infrastructure no? As we would need a centralized seedbox that would provide the cache in a scenario where the seeds lost an interest.

MatthewCroughan commented 3 months ago

Who is going to write the code, and how are they going to eat?

Kreyren commented 3 months ago

Who is going to write the code, and how are they going to eat? -- @MatthewCroughan (https://github.com/Kreyren/kreyren/issues/110#issuecomment-2294255144)

Sorry if it sounded disrespectful it wasn't meant to be. For clarification the conversation was sparked by GitHub's outage and the amount of resources that NixOS cache needs to operate in the light of https://discourse.nixos.org/t/the-nixos-foundations-call-to-action-s3-costs-require-community-support/28672.

So i am currently trying to better understand the technical side of changing the cache distribution on BitTorrent-based and managing the issue of getting the torrents set up to avoid a scenario where user tries to pull results that are not available as afaik the only sane way is to handle that is to have a centralized seedbox that provides them so that even if there are no seeds the user can always pull them from there? As that was a breaking issue that eventually killed basically the same solution on debian in ~2013 among other issues mentioned in the reference, though it seems that nix's implementation manages nearly all of them? (establish a feasibility)

In terms of grant if you want to work on this then i might be able to ask for a grant for you.

MatthewCroughan commented 3 months ago

Sorry if it sounded disrespectful it wasn't meant to be

It doesn't at all.

MatthewCroughan commented 3 months ago

In terms of grant if you want to work on this then i might be able to ask for a grant for you.

If there's an open source project that can pay just about anything, I'm happy to be involved in any capacity. But I can only afford to do things like this for free if they provide immediate satisfaction and are short-term. This is definitely a long term thing with quite a lot of work involved, and even if I enjoy it, I can't do it without damaging myself.