search

zhaofengli / attic

Multi-tenant Nix Binary Cache
https://docs.attic.rs
Other
1.02k stars 77 forks source link

Big uploads seem to fail. #141

Open MGlolenstine opened 4 months ago

MGlolenstine commented 4 months ago

Whenever I try to upload larger files that have been added to the /nix/store (I need to package a proprietary project for our projects, so I'm unable to redistribute), which I add by manually downloading and then running nix store add-file ~/Downloads/<file>, the file weighs about 980MB.

I tried pushing it using attic push work /nix/store/<path>, but it fails to upload (with SQLite, it failed at 10MB, with PostgreSQL it fails at 250MB).

Log:

⚙️ Pushing 1 paths to "work" on "cache" (0 already cached, 0 in upstream)...
❌ pks4344w4p8syzafh5bys05dn2pj6k9s-en.st-stm32cubeide_1.15.1_21094_20240412_1041_amd64.deb_bundle.sh.zip: request or response body error: error reading a body from connection: stream error received: not a result of an error
Error: request or response body error: error reading a body from connection: stream error received: not a result of an error

Caused by:
    0: error reading a body from connection: stream error received: not a result of an error
    1: stream error received: not a result of an error

I can see that the database is getting spammed with the insert and update requests, so I guess that's where the difference between SQLite and PostgreSQL comes in.

I'm wondering if there's something else I could do to improve the performance and allow this to be uploaded.

Nebucatnetzer commented 4 months ago

Looks like it is the same issue as this one: https://github.com/zhaofengli/attic/issues/24

MGlolenstine commented 4 months ago

While it does look similar at the first glance, they seem to get errors inside of attic logs, where I don't get any and the error, while occurring in both issues, seem to differ a bit.

gmacon commented 3 months ago

The "not a result of an error" is a symptom of https://github.com/hyperium/hyper/issues/2500. I noticed this error with an attic instance deployed in a kubernetes cluster and behind the nginx ingress since nginx sends a GOAWAY after handling a certain number of requests.

Nebucatnetzer commented 3 months ago

Does it work with Caddy? I don’t have the possibility to test it myself for the next few days but I will if no one else does.

gmacon commented 3 months ago

I don't actually control this kubernetes cluster, so I don't have an easy way to test it myself. I would say "This behavior is within-spec, and attic should handle it." except that that it looks to me like reqwest doesn't expose sufficient detail to make that happen, and handling it today would require using hyper directly.

Svenum commented 1 week ago

Maybe the same as: https://github.com/zhaofengli/attic/issues/170