search

flokli / nix-casync

A more efficient way to store and substitute Nix store paths
80 stars 4 forks source link

Support compression while uploading #18

Closed flokli closed 2 years ago

flokli commented 2 years ago

Closes #16.

This will change the HTTP interface slightly - after uploading a /nar/$filehash.nar.$suffix, that file will be available at /nar/$narhash.nar, in uncompressed form.

Uploaded .narinfo files are rewritten, so they always point to /nar/$narhash.nar, with updated FileSize and FileHash (copied from NarSize/NarHash.

~This needs a bunch of more tests, and doc updates before merging.~

Closes #17.

bbigras commented 2 years ago

with zstd I got this:

❯ nix copy \
  --to "http://10.1.10.99:9000?compression=zstd&secret-key=/home/bbigras/casync-keys-other/cache-priv-key.pem" /nix/store/nfw1qqdi7rpshfkrn3nqf4k6zrhcdkqr-gcc-10.3.0
warning: error: unable to upload 'http://10.1.10.99:9000/nar/0m8axs4cgj8prkqfkfl8w7bqvxxlkvs0jl05fhd6a56wgqc5h4xl.nar.zst': HTTP error 500 ('Internal Server Error')

       response body:

       PUT handle-nar: failed to read from underlying reader: read tcp 10.1.10.99:9000->100.67.149.7:55078: i/o timeout; retrying in 258 ms
warning: error: unable to upload 'http://10.1.10.99:9000/nar/0m8axs4cgj8prkqfkfl8w7bqvxxlkvs0jl05fhd6a56wgqc5h4xl.nar.zst': HTTP error 100 ('Continue') (curl error: Failed sending data to the peer)

       response body:

       PUT handle-nar: failed to read from underlying reader: read tcp 10.1.10.99:9000->100.67.149.7:55078: i/o timeout; retrying in 516 ms
warning: error: unable to upload 'http://10.1.10.99:9000/nar/0m8axs4cgj8prkqfkfl8w7bqvxxlkvs0jl05fhd6a56wgqc5h4xl.nar.zst': HTTP error 100 ('Continue') (curl error: Failed sending data to the peer)

       response body:

       PUT handle-nar: failed to read from underlying reader: read tcp 10.1.10.99:9000->100.67.149.7:55078: i/o timeout; retrying in 1207 ms
warning: error: unable to upload 'http://10.1.10.99:9000/nar/0m8axs4cgj8prkqfkfl8w7bqvxxlkvs0jl05fhd6a56wgqc5h4xl.nar.zst': HTTP error 100 ('Continue') (curl error: Failed sending data to the peer)

       response body:

       PUT handle-nar: failed to read from underlying reader: read tcp 10.1.10.99:9000->100.67.149.7:55078: i/o timeout; retrying in 2047 ms
error: while uploading to HTTP binary cache at 'http://10.1.10.99:9000': error: unable to upload 'http://10.1.10.99:9000/nar/0m8axs4cgj8prkqfkfl8w7bqvxxlkvs0jl05fhd6a56wgqc5h4xl.nar.zst': HTTP error 100 ('Continue') (curl error: Failed sending data to the peer)

              response body:

              PUT handle-nar: failed to read from underlying reader: read tcp 10.1.10.99:9000->100.67.149.7:55078: i/o timeout

server side

❯ ./nix_casync serve --cache-path=/home/bbigras/casync                                                                                                                                                       
INFO[0000] Starting Server at [::]:9000
2021/12/12 03:17:45 "HEAD http://10.1.10.99:9000/nar/0m8axs4cgj8prkqfkfl8w7bqvxxlkvs0jl05fhd6a56wgqc5h4xl.nar.zst HTTP/1.1" from 100.67.149.7:55078 - 404 35B in 174.951µs
2021/12/12 03:18:35 "PUT http://10.1.10.99:9000/nar/0m8axs4cgj8prkqfkfl8w7bqvxxlkvs0jl05fhd6a56wgqc5h4xl.nar.zst HTTP/1.1" from 100.67.149.7:55078 - 500 113B in 50.409619353s
2021/12/12 03:18:36 "PUT http://10.1.10.99:9000/nar/0m8axs4cgj8prkqfkfl8w7bqvxxlkvs0jl05fhd6a56wgqc5h4xl.nar.zst HTTP/1.1" from 100.67.149.7:55080 - 500 63B in 176.654168ms
2021/12/12 03:18:36 http: panic serving 100.67.149.7:55080: runtime error: invalid memory address or nil pointer dereference
goroutine 35 [running]:
net/http.(*conn).serve.func1(0xc000428140)
        /nix/store/5zvhj5hvy9mpgr7h8bjw3hj4jfnfd9zh-go-1.16.10/share/go/src/net/http/server.go:1804 +0x153
panic(0xb40d60, 0x1045ef0)
        /nix/store/5zvhj5hvy9mpgr7h8bjw3hj4jfnfd9zh-go-1.16.10/share/go/src/runtime/panic.go:971 +0x499
os.(*File).Name(...)
        /nix/store/5zvhj5hvy9mpgr7h8bjw3hj4jfnfd9zh-go-1.16.10/share/go/src/os/file.go:55
github.com/flokli/nix-casync/pkg/store/narstore.(*casyncStoreNarWriter).Close(0xc000492000, 0x0, 0x0)
        /home/bbigras/go/src/github.com/flokli/nix-casync/pkg/store/narstore/casync_store_nar_writer.go:61 +0x64
github.com/flokli/nix-casync/pkg/server.(*Server).handleNar(0xc0003f64c0, 0x7f816b7cc2b8, 0xc0000ee380, 0xc000490100)
        /home/bbigras/go/src/github.com/flokli/nix-casync/pkg/server/server.go:179 +0xb9f
net/http.HandlerFunc.ServeHTTP(0xc0003c4af0, 0x7f816b7cc2b8, 0xc0000ee380, 0xc000490100)
        /nix/store/5zvhj5hvy9mpgr7h8bjw3hj4jfnfd9zh-go-1.16.10/share/go/src/net/http/server.go:2049 +0x44
github.com/go-chi/chi/v5.(*Mux).routeHTTP(0xc0003f44e0, 0x7f816b7cc2b8, 0xc0000ee380, 0xc000490100)
        /home/bbigras/go/pkg/mod/github.com/go-chi/chi/v5@v5.0.7/mux.go:442 +0x2a9
net/http.HandlerFunc.ServeHTTP(0xc0003c4970, 0x7f816b7cc2b8, 0xc0000ee380, 0xc000490100)
        /nix/store/5zvhj5hvy9mpgr7h8bjw3hj4jfnfd9zh-go-1.16.10/share/go/src/net/http/server.go:2049 +0x44
github.com/go-chi/chi/v5.(*Mux).ServeHTTP(0xc0003f44e0, 0x7f816b7cc2b8, 0xc0000ee380, 0xc000490000)
        /home/bbigras/go/pkg/mod/github.com/go-chi/chi/v5@v5.0.7/mux.go:88 +0x310
github.com/go-chi/chi/middleware.RequestLogger.func1.1(0xce0750, 0xc0003a4000, 0xc00015e000)
        /home/bbigras/go/pkg/mod/github.com/go-chi/chi@v1.5.4/middleware/logger.go:57 +0x2e2
net/http.HandlerFunc.ServeHTTP(0xc00042a150, 0xce0750, 0xc0003a4000, 0xc00015e000)
        /nix/store/5zvhj5hvy9mpgr7h8bjw3hj4jfnfd9zh-go-1.16.10/share/go/src/net/http/server.go:2049 +0x44
net/http.serverHandler.ServeHTTP(0xc0003a4620, 0xce0750, 0xc0003a4000, 0xc00015e000)
        /nix/store/5zvhj5hvy9mpgr7h8bjw3hj4jfnfd9zh-go-1.16.10/share/go/src/net/http/server.go:2867 +0xa3
net/http.(*conn).serve(0xc000428140, 0xce1b40, 0xc0003f6080)
        /nix/store/5zvhj5hvy9mpgr7h8bjw3hj4jfnfd9zh-go-1.16.10/share/go/src/net/http/server.go:1932 +0x8cd
created by net/http.(*Server).Serve
        /nix/store/5zvhj5hvy9mpgr7h8bjw3hj4jfnfd9zh-go-1.16.10/share/go/src/net/http/server.go:2993 +0x39b
2021/12/12 03:18:36 "PUT http://10.1.10.99:9000/nar/0m8axs4cgj8prkqfkfl8w7bqvxxlkvs0jl05fhd6a56wgqc5h4xl.nar.zst HTTP/1.1" from 100.67.149.7:55082 - 500 63B in 178.552046ms
2021/12/12 03:18:36 http: panic serving 100.67.149.7:55082: runtime error: invalid memory address or nil pointer dereference
goroutine 6 [running]:
net/http.(*conn).serve.func1(0xc0000c2a00)
        /nix/store/5zvhj5hvy9mpgr7h8bjw3hj4jfnfd9zh-go-1.16.10/share/go/src/net/http/server.go:1804 +0x153
panic(0xb40d60, 0x1045ef0)
        /nix/store/5zvhj5hvy9mpgr7h8bjw3hj4jfnfd9zh-go-1.16.10/share/go/src/runtime/panic.go:971 +0x499
os.(*File).Name(...)
        /nix/store/5zvhj5hvy9mpgr7h8bjw3hj4jfnfd9zh-go-1.16.10/share/go/src/os/file.go:55
github.com/flokli/nix-casync/pkg/store/narstore.(*casyncStoreNarWriter).Close(0xc000492200, 0x0, 0x0)
        /home/bbigras/go/src/github.com/flokli/nix-casync/pkg/store/narstore/casync_store_nar_writer.go:61 +0x64
github.com/flokli/nix-casync/pkg/server.(*Server).handleNar(0xc0003f64c0, 0x7f816b7cc2b8, 0xc0000ee5c0, 0xc000490400)
        /home/bbigras/go/src/github.com/flokli/nix-casync/pkg/server/server.go:179 +0xb9f
net/http.HandlerFunc.ServeHTTP(0xc0003c4af0, 0x7f816b7cc2b8, 0xc0000ee5c0, 0xc000490400)
        /nix/store/5zvhj5hvy9mpgr7h8bjw3hj4jfnfd9zh-go-1.16.10/share/go/src/net/http/server.go:2049 +0x44
github.com/go-chi/chi/v5.(*Mux).routeHTTP(0xc0003f44e0, 0x7f816b7cc2b8, 0xc0000ee5c0, 0xc000490400)
        /home/bbigras/go/pkg/mod/github.com/go-chi/chi/v5@v5.0.7/mux.go:442 +0x2a9
net/http.HandlerFunc.ServeHTTP(0xc0003c4970, 0x7f816b7cc2b8, 0xc0000ee5c0, 0xc000490400)
        /nix/store/5zvhj5hvy9mpgr7h8bjw3hj4jfnfd9zh-go-1.16.10/share/go/src/net/http/server.go:2049 +0x44
github.com/go-chi/chi/v5.(*Mux).ServeHTTP(0xc0003f44e0, 0x7f816b7cc2b8, 0xc0000ee5c0, 0xc000490300)
        /home/bbigras/go/pkg/mod/github.com/go-chi/chi/v5@v5.0.7/mux.go:88 +0x310
github.com/go-chi/chi/middleware.RequestLogger.func1.1(0xce0750, 0xc0003a40e0, 0xc000490200)
        /home/bbigras/go/pkg/mod/github.com/go-chi/chi@v1.5.4/middleware/logger.go:57 +0x2e2
net/http.HandlerFunc.ServeHTTP(0xc00042a150, 0xce0750, 0xc0003a40e0, 0xc000490200)
        /nix/store/5zvhj5hvy9mpgr7h8bjw3hj4jfnfd9zh-go-1.16.10/share/go/src/net/http/server.go:2049 +0x44
net/http.serverHandler.ServeHTTP(0xc0003a4620, 0xce0750, 0xc0003a40e0, 0xc000490200)
        /nix/store/5zvhj5hvy9mpgr7h8bjw3hj4jfnfd9zh-go-1.16.10/share/go/src/net/http/server.go:2867 +0xa3
net/http.(*conn).serve(0xc0000c2a00, 0xce1b40, 0xc0000ee3c0)
        /nix/store/5zvhj5hvy9mpgr7h8bjw3hj4jfnfd9zh-go-1.16.10/share/go/src/net/http/server.go:1932 +0x8cd
created by net/http.(*Server).Serve
        /nix/store/5zvhj5hvy9mpgr7h8bjw3hj4jfnfd9zh-go-1.16.10/share/go/src/net/http/server.go:2993 +0x39b
2021/12/12 03:18:38 "PUT http://10.1.10.99:9000/nar/0m8axs4cgj8prkqfkfl8w7bqvxxlkvs0jl05fhd6a56wgqc5h4xl.nar.zst HTTP/1.1" from 100.67.149.7:55084 - 500 63B in 177.075238ms
2021/12/12 03:18:38 http: panic serving 100.67.149.7:55084: runtime error: invalid memory address or nil pointer dereference
goroutine 7 [running]:
net/http.(*conn).serve.func1(0xc0000c2dc0)
        /nix/store/5zvhj5hvy9mpgr7h8bjw3hj4jfnfd9zh-go-1.16.10/share/go/src/net/http/server.go:1804 +0x153
panic(0xb40d60, 0x1045ef0)
        /nix/store/5zvhj5hvy9mpgr7h8bjw3hj4jfnfd9zh-go-1.16.10/share/go/src/runtime/panic.go:971 +0x499
os.(*File).Name(...)
        /nix/store/5zvhj5hvy9mpgr7h8bjw3hj4jfnfd9zh-go-1.16.10/share/go/src/os/file.go:55
github.com/flokli/nix-casync/pkg/store/narstore.(*casyncStoreNarWriter).Close(0xc00015c080, 0x0, 0x0)
        /home/bbigras/go/src/github.com/flokli/nix-casync/pkg/store/narstore/casync_store_nar_writer.go:61 +0x64
github.com/flokli/nix-casync/pkg/server.(*Server).handleNar(0xc0003f64c0, 0x7f816b7cc2b8, 0xc0003f62c0, 0xc00015e300)
        /home/bbigras/go/src/github.com/flokli/nix-casync/pkg/server/server.go:179 +0xb9f
net/http.HandlerFunc.ServeHTTP(0xc0003c4af0, 0x7f816b7cc2b8, 0xc0003f62c0, 0xc00015e300)
        /nix/store/5zvhj5hvy9mpgr7h8bjw3hj4jfnfd9zh-go-1.16.10/share/go/src/net/http/server.go:2049 +0x44
github.com/go-chi/chi/v5.(*Mux).routeHTTP(0xc0003f44e0, 0x7f816b7cc2b8, 0xc0003f62c0, 0xc00015e300)
        /home/bbigras/go/pkg/mod/github.com/go-chi/chi/v5@v5.0.7/mux.go:442 +0x2a9
net/http.HandlerFunc.ServeHTTP(0xc0003c4970, 0x7f816b7cc2b8, 0xc0003f62c0, 0xc00015e300)
        /nix/store/5zvhj5hvy9mpgr7h8bjw3hj4jfnfd9zh-go-1.16.10/share/go/src/net/http/server.go:2049 +0x44
github.com/go-chi/chi/v5.(*Mux).ServeHTTP(0xc0003f44e0, 0x7f816b7cc2b8, 0xc0003f62c0, 0xc00015e200)
        /home/bbigras/go/pkg/mod/github.com/go-chi/chi/v5@v5.0.7/mux.go:88 +0x310
github.com/go-chi/chi/middleware.RequestLogger.func1.1(0xce0750, 0xc000346000, 0xc00015e100)
        /home/bbigras/go/pkg/mod/github.com/go-chi/chi@v1.5.4/middleware/logger.go:57 +0x2e2
net/http.HandlerFunc.ServeHTTP(0xc00042a150, 0xce0750, 0xc000346000, 0xc00015e100)
        /nix/store/5zvhj5hvy9mpgr7h8bjw3hj4jfnfd9zh-go-1.16.10/share/go/src/net/http/server.go:2049 +0x44
net/http.serverHandler.ServeHTTP(0xc0003a4620, 0xce0750, 0xc000346000, 0xc00015e100)
        /nix/store/5zvhj5hvy9mpgr7h8bjw3hj4jfnfd9zh-go-1.16.10/share/go/src/net/http/server.go:2867 +0xa3
net/http.(*conn).serve(0xc0000c2dc0, 0xce1b40, 0xc0003f6140)
        /nix/store/5zvhj5hvy9mpgr7h8bjw3hj4jfnfd9zh-go-1.16.10/share/go/src/net/http/server.go:1932 +0x8cd
created by net/http.(*Server).Serve
        /nix/store/5zvhj5hvy9mpgr7h8bjw3hj4jfnfd9zh-go-1.16.10/share/go/src/net/http/server.go:2993 +0x39b
2021/12/12 03:18:40 "PUT http://10.1.10.99:9000/nar/0m8axs4cgj8prkqfkfl8w7bqvxxlkvs0jl05fhd6a56wgqc5h4xl.nar.zst HTTP/1.1" from 100.67.149.7:55086 - 500 63B in 177.228687ms
2021/12/12 03:18:40 http: panic serving 100.67.149.7:55086: runtime error: invalid memory address or nil pointer dereference
goroutine 32 [running]:
net/http.(*conn).serve.func1(0xc0001a0a00)
        /nix/store/5zvhj5hvy9mpgr7h8bjw3hj4jfnfd9zh-go-1.16.10/share/go/src/net/http/server.go:1804 +0x153
panic(0xb40d60, 0x1045ef0)
        /nix/store/5zvhj5hvy9mpgr7h8bjw3hj4jfnfd9zh-go-1.16.10/share/go/src/runtime/panic.go:971 +0x499
os.(*File).Name(...)
        /nix/store/5zvhj5hvy9mpgr7h8bjw3hj4jfnfd9zh-go-1.16.10/share/go/src/os/file.go:55
github.com/flokli/nix-casync/pkg/store/narstore.(*casyncStoreNarWriter).Close(0xc000464080, 0x0, 0x0)
        /home/bbigras/go/src/github.com/flokli/nix-casync/pkg/store/narstore/casync_store_nar_writer.go:61 +0x64
github.com/flokli/nix-casync/pkg/server.(*Server).handleNar(0xc0003f64c0, 0x7f816b7cc2b8, 0xc00046a1c0, 0xc00046e200)
        /home/bbigras/go/src/github.com/flokli/nix-casync/pkg/server/server.go:179 +0xb9f
net/http.HandlerFunc.ServeHTTP(0xc0003c4af0, 0x7f816b7cc2b8, 0xc00046a1c0, 0xc00046e200)
        /nix/store/5zvhj5hvy9mpgr7h8bjw3hj4jfnfd9zh-go-1.16.10/share/go/src/net/http/server.go:2049 +0x44
github.com/go-chi/chi/v5.(*Mux).routeHTTP(0xc0003f44e0, 0x7f816b7cc2b8, 0xc00046a1c0, 0xc00046e200)
        /home/bbigras/go/pkg/mod/github.com/go-chi/chi/v5@v5.0.7/mux.go:442 +0x2a9
net/http.HandlerFunc.ServeHTTP(0xc0003c4970, 0x7f816b7cc2b8, 0xc00046a1c0, 0xc00046e200)
        /nix/store/5zvhj5hvy9mpgr7h8bjw3hj4jfnfd9zh-go-1.16.10/share/go/src/net/http/server.go:2049 +0x44
github.com/go-chi/chi/v5.(*Mux).ServeHTTP(0xc0003f44e0, 0x7f816b7cc2b8, 0xc00046a1c0, 0xc00046e100)
        /home/bbigras/go/pkg/mod/github.com/go-chi/chi/v5@v5.0.7/mux.go:88 +0x310
github.com/go-chi/chi/middleware.RequestLogger.func1.1(0xce0750, 0xc00047c000, 0xc00046e000)
        /home/bbigras/go/pkg/mod/github.com/go-chi/chi@v1.5.4/middleware/logger.go:57 +0x2e2
net/http.HandlerFunc.ServeHTTP(0xc00042a150, 0xce0750, 0xc00047c000, 0xc00046e000)
        /nix/store/5zvhj5hvy9mpgr7h8bjw3hj4jfnfd9zh-go-1.16.10/share/go/src/net/http/server.go:2049 +0x44
net/http.serverHandler.ServeHTTP(0xc0003a4620, 0xce0750, 0xc00047c000, 0xc00046e000)
        /nix/store/5zvhj5hvy9mpgr7h8bjw3hj4jfnfd9zh-go-1.16.10/share/go/src/net/http/server.go:2867 +0xa3
net/http.(*conn).serve(0xc0001a0a00, 0xce1b40, 0xc00046a000)
        /nix/store/5zvhj5hvy9mpgr7h8bjw3hj4jfnfd9zh-go-1.16.10/share/go/src/net/http/server.go:1932 +0x8cd
created by net/http.(*Server).Serve
        /nix/store/5zvhj5hvy9mpgr7h8bjw3hj4jfnfd9zh-go-1.16.10/share/go/src/net/http/server.go:2993 +0x39b
flokli commented 2 years ago

@bbigras thanks for the report!

This was happening when running Close() on the casyncStoreNarWriter without any byte written into it so far.

I had some buggy init() function, which I now removed, and added a regression test.

bbigras commented 2 years ago

Now I have this:

with ce00255041d579586a82569e0dd8e049743f84cc

❯ ~/upload-cache.sh
warning: Git tree '/home/bbigras/dev/my_test' is dirty
warning: error: unable to upload 'http://10.1.10.99:9000/nar/0m8axs4cgj8prkqfkfl8w7bqvxxlkvs0jl05fhd6a56wgqc5h4xl.nar.zst': HTTP error 500 ('Internal Server Error')

       response body:

       PUT handle-nar: failed to read from underlying reader: read tcp 10.1.10.99:9000->100.67.149.7:57806: i/o timeout; retrying in 286 ms
warning: error: unable to upload 'http://10.1.10.99:9000/nar/0m8axs4cgj8prkqfkfl8w7bqvxxlkvs0jl05fhd6a56wgqc5h4xl.nar.zst': HTTP error 500 ('Internal Server Error')

       response body:

       PUT handle-nar: failed to read from underlying reader: read tcp 10.1.10.99:9000->100.67.149.7:57806: i/o timeout
       PUT handle-nar: failed to decompress: Unknown frame descriptor; retrying in 687 ms
warning: error: unable to upload 'http://10.1.10.99:9000/nar/0m8axs4cgj8prkqfkfl8w7bqvxxlkvs0jl05fhd6a56wgqc5h4xl.nar.zst': HTTP error 500 ('Internal Server Error')

       response body:

       PUT handle-nar: failed to read from underlying reader: read tcp 10.1.10.99:9000->100.67.149.7:57806: i/o timeout
       PUT handle-nar: failed to decompress: Unknown frame descriptor
       PUT handle-nar: failed to decompress: Unknown frame descriptor; retrying in 1276 ms
warning: error: unable to upload 'http://10.1.10.99:9000/nar/0m8axs4cgj8prkqfkfl8w7bqvxxlkvs0jl05fhd6a56wgqc5h4xl.nar.zst': HTTP error 500 ('Internal Server Error')

       response body:

       PUT handle-nar: failed to read from underlying reader: read tcp 10.1.10.99:9000->100.67.149.7:57806: i/o timeout
       PUT handle-nar: failed to decompress: Unknown frame descriptor
       PUT handle-nar: failed to decompress: Unknown frame descriptor
       PUT handle-nar: failed to decompress: Unknown frame descriptor; retrying in 2094 ms
error: while uploading to HTTP binary cache at 'http://10.1.10.99:9000': error: unable to upload 'http://10.1.10.99:9000/nar/0m8axs4cgj8prkqfkfl8w7bqvxxlkvs0jl05fhd6a56wgqc5h4xl.nar.zst': HTTP error 500 ('Internal Server Error')

              response body:

              PUT handle-nar: failed to read from underlying reader: read tcp 10.1.10.99:9000->100.67.149.7:57806: i/o timeout
              PUT handle-nar: failed to decompress: Unknown frame descriptor
              PUT handle-nar: failed to decompress: Unknown frame descriptor
              PUT handle-nar: failed to decompress: Unknown frame descriptor
              PUT handle-nar: failed to decompress: Unknown frame descriptor

server

2021/12/12 17:51:22 "HEAD http://10.1.10.99:9000/nar/0m8axs4cgj8prkqfkfl8w7bqvxxlkvs0jl05fhd6a56wgqc5h4xl.nar.zst HTTP/1.1" from 100.67.149.7:57806 - 404 35B in 136.63µs
2021/12/12 17:52:13 "PUT http://10.1.10.99:9000/nar/0m8axs4cgj8prkqfkfl8w7bqvxxlkvs0jl05fhd6a56wgqc5h4xl.nar.zst HTTP/1.1" from 100.67.149.7:57806 - 500 113B in 50.324119842s
2021/12/12 17:52:13 "PUT http://10.1.10.99:9000/nar/0m8axs4cgj8prkqfkfl8w7bqvxxlkvs0jl05fhd6a56wgqc5h4xl.nar.zst HTTP/1.1" from 100.67.149.7:57808 - 500 63B in 178.731122ms
2021/12/12 17:52:14 "PUT http://10.1.10.99:9000/nar/0m8axs4cgj8prkqfkfl8w7bqvxxlkvs0jl05fhd6a56wgqc5h4xl.nar.zst HTTP/1.1" from 100.67.149.7:57812 - 500 63B in 179.255748ms
2021/12/12 17:52:16 "PUT http://10.1.10.99:9000/nar/0m8axs4cgj8prkqfkfl8w7bqvxxlkvs0jl05fhd6a56wgqc5h4xl.nar.zst HTTP/1.1" from 100.67.149.7:57814 - 500 63B in 177.852866ms
2021/12/12 17:52:18 "PUT http://10.1.10.99:9000/nar/0m8axs4cgj8prkqfkfl8w7bqvxxlkvs0jl05fhd6a56wgqc5h4xl.nar.zst HTTP/1.1" from 100.67.149.7:57816 - 500 63B in 179.330727ms
bbigras commented 2 years ago

same thing with cc0b3b02c5af4bbb6d41af7815cba247b0fe755a

flokli commented 2 years ago

same thing with cc0b3b0

I can't reproduce, I was able to nix copy --extra-experimental-features nix-command --to "http://localhost:9000?compression=zstd" /run/current-system just fine.

What version of Nix is this? failed to decompress: Unknown frame descriptor suggests the zstd payload is garbled…

flokli commented 2 years ago

https://github.com/DataDog/zstd/issues/81#issuecomment-625951583 suggests this is related to added padding at the end of the frame. Maybe Nix 2.4 (which switched to libarchive) doesn't add this kind of padding?

flokli commented 2 years ago

Let's merge this in, also as it contains fixes for https://github.com/flokli/nix-casync/issues/17.

If the zstd compressor turns out to be problematic, we can probably switch it out, or remove it from the list of supported uploaded narfiles.

flokli commented 2 years ago

Recap on the issues @bbigras observed: We debugged this a bit further, and it seems something in the setup was rewriting HTTP requests, while it crossed the network.

If HTTPs was added to the mix, or the copy command run locally, it didn't appear.