search

tus / tusd

Reference server implementation in Go of tus: the open protocol for resumable file uploads
https://tus.github.io/tusd
MIT License
3.05k stars 475 forks source link

panic serving 127.0.0.1:****: unaligned 64-bit atomic operation #1047

Closed martnst closed 9 months ago

martnst commented 9 months ago

Describe the bug File Upload not working on Debian Installation.

I did install the the binary from https://github.com/tus/tusd/releases/download/v2.2.1/tusd_linux_386.tar.gz on a Debian server.

Attempting to upload file I get this error: 

http: panic serving 127.0.0.1:43434: unaligned 64-bit atomic operation
goroutine 36 [running]:
net/http.(*conn).serve.func1()
\tnet/http/server.go:1868 +0xe0
panic({0x8c538e0, 0x8f80608})
\truntime/panic.go:920 +0x24c
runtime/internal/atomic.panicUnaligned()
\truntime/internal/atomic/unaligned.go:8 +0x2d
runtime/internal/atomic.Xadd64(0xb3091d4, 0x12)
\truntime/internal/atomic/atomic_386.s:125 +0x11
github.com/tus/tusd/v2/pkg/handler.(*bodyReader).Read(0xb3091c0, {0xb338000, 0x8000, 0x8000})
\tgithub.com/tus/tusd/v2/pkg/handler/body_reader.go:43 +0x9a
io.copyBuffer({0x8f811ac, 0xb27df98}, {0x8f81314, 0xb3091c0}, {0x0, 0x0, 0x0})
\tio/io.go:430 +0x1bf
io.Copy(...)
\tio/io.go:389
os.genericReadFrom(0xb27df98, {0x8f81314, 0xb3091c0})
\tos/file.go:161 +0x53
os.(*File).ReadFrom(0xb27df98, {0x8f81314, 0xb3091c0})
\tos/file.go:155 +0xda
io.copyBuffer({0x8f80f40, 0xb27df98}, {0x8f81314, 0xb3091c0}, {0x0, 0x0, 0x0})
\tio/io.go:416 +0x127
io.Copy(...)
\tio/io.go:389
github.com/tus/tusd/v2/pkg/filestore.(*fileUpload).WriteChunk(0xb1ab9f0, {0x8f8c5e0, 0xb1b3200}, 0x0, {0x8f81314, 0xb3091c0})
\tgithub.com/tus/tusd/v2/pkg/filestore/filestore.go:165 +0x90
github.com/tus/tusd/v2/pkg/handler.(*UnroutedHandler).writeChunk(0xb3ee000, 0xb1b3200, {0xcc, {0x0, 0x0}, 0xb3091a0}, {0x8f8c5a0, 0xb1ab9f0}, {{0xb1977e0, 0x20}, ...})
\tgithub.com/tus/tusd/v2/pkg/handler/unrouted_handler.go:874 +0x4d9
github.com/tus/tusd/v2/pkg/handler.(*UnroutedHandler).PatchFile(0xb3ee000, {0x8f8bc88, 0xb1c8320}, 0xb19ce80)
\tgithub.com/tus/tusd/v2/pkg/handler/unrouted_handler.go:770 +0x8b9
net/http.HandlerFunc.ServeHTTP(0xb04b2f0, {0x8f8bc88, 0xb1c8320}, 0xb19ce80)
\tnet/http/server.go:2136 +0x34
github.com/bmizerany/pat.(*PatternServeMux).ServeHTTP(0xb011500, {0x8f8bc88, 0xb1c8320}, 0xb19ce80)
\tgithub.com/bmizerany/pat@v0.0.0-20170815010413-6226ea591a40/mux.go:117 +0x172
github.com/tus/tusd/v2/pkg/handler.NewHandler.(*UnroutedHandler).Middleware.func1({0x8f8bc88, 0xb1c8320}, 0xb19ce00)
\tgithub.com/tus/tusd/v2/pkg/handler/unrouted_handler.go:252 +0x943
net/http.HandlerFunc.ServeHTTP(0xb011510, {0x8f8bc88, 0xb1c8320}, 0xb19ce00)
\tnet/http/server.go:2136 +0x34
github.com/tus/tusd/v2/cmd/tusd/cli.Serve.StripPrefix.func9({0x8f8bc88, 0xb1c8320}, 0xb19cd00)
\tnet/http/server.go:2179 +0x20b
net/http.HandlerFunc.ServeHTTP(0xb0ad398, {0x8f8bc88, 0xb1c8320}, 0xb19cd00)
\tnet/http/server.go:2136 +0x34
net/http.(*ServeMux).ServeHTTP(0xb03ad80, {0x8f8bc88, 0xb1c8320}, 0xb19cd00)
\tnet/http/server.go:2514 +0x15e
net/http.serverHandler.ServeHTTP({0xb3ee0a0}, {0x8f8bc88, 0xb1c8320}, 0xb19cd00)
\tnet/http/server.go:2938 +0x99
net/http.(*conn).serve(0xb30d740, {0x8f8c480, 0xb0ad680})
\tnet/http/server.go:2009 +0x66d
created by net/http.(*Server).Serve in goroutine 1
\tnet/http/server.go:3086 +0x47f

To Reproduce Steps to reproduce the behavior:

  1. Execute command '...'
  2. Start upload '....'
  3. See error

Expected behavior Request should work.

Setup details Please provide following details, if applicable to your situation:

martnst commented 9 months ago

Here is how I installed it:

  1. Create & enter a temporary working folder

    mkdir tmp-install-tusd && cd tmp-install-tusd

  2. Download 

    wget https://github.com/tus/tusd/releases/download/v2.2.0/tusd_linux_386.tar.gz

  3. Verify integrity of the file

    curl -L -s https://github.com/tus/tusd/releases/download/v2.2.0/tusd_linux_386.tar.gz.sha256 | shasum -a 256 -c
tusd_linux_386.tar.gz: OK

  4. Unpack the archive 

    tar xzf tusd_linux_386.tar.gz

  5. Move the binary 

    cp -v tusd_linux_386/tusd /usr/local/bin/
'tusd_linux_386/tusd' -> '/usr/local/bin/tusd'

  6. Test the server

    tsud

    Output should look like this:

    2023/12/13 11:29:31.164283 Using '/root/data' as directory storage.
2023/12/13 11:29:31.164608 Using 0.00MB as maximum size.
2023/12/13 11:29:31.164635 Supported tus extensions: creation,creation-with-upload,termination,concatenation,creation-defer-length
2023/12/13 11:29:31.164640 Using 0.0.0.0:8080 as address to listen.
2023/12/13 11:29:31.164645 Using /files/ as the base path.
2023/12/13 11:29:31.164678 Using /metrics as the metrics path.
2023/12/13 11:29:31.164860 You can now upload files to: http://[::]:8080/files/

  7. Remove the working folder 

    cd .. && rm -rfv tmp-install-tusd
Acconut commented 9 months ago

Thanks for the detailed report, that's very helpful!

You seem to be on a 64-bit machine (x84_64) but downloaded a 32-bit build of tusd (386 architecture). Was there a particular reason why you downloaded tusd_linux_386.tar.gz and not tusd_linux_amd64.tar.gz?

I found similar reports where this happened when the Go code was compiled for 32-bit architecture, e.g. https://www.couchbase.com/forums/t/go-sdk-panics-with-unaligned-64-bit-atomic-operation-on-x86-64-ubuntu-server/34124/2

The panic is caused by https://github.com/tus/tusd/blob/f03aa9b4ddd69cf83c1fa3f74f1172c179be7bdc/pkg/handler/body_reader.go#L43

We should be able to fix this by adjusting the position of the field that is written to like they did in https://github.com/blevesearch/zapx/pull/148.

martnst commented 9 months ago

Ah right. Thanks a lot. That was a stupid mistake of mine. I am not working with linux on a daily basis.