search

anacrolix / torrent

Full-featured BitTorrent client package and utilities
Mozilla Public License 2.0
5.51k stars 622 forks source link

invalid memory address or nil pointer dereference in request-strategy.(*PieceRequestOrder).Update #795

Closed vintikzzz closed 1 year ago

vintikzzz commented 1 year ago

Hi!

Got this today:

2022-12-22T06:46:18+0000 ERR [main client github.com/anacrolix/torrent.(*Peer).receiveChunk:1526]: *torrent.PeerConn 0xc02959a900 [id="-UT360W-\xfe\xb5\xe6y\xc9_\xd0\xe4JVy\x84", exts=0000000000100005, v="μTorrent 3.6"]: writing received chunk 484874: short write
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x10 pc=0xc69ad5]

goroutine 12618781 [running]:
github.com/anacrolix/torrent/request-strategy.(*PieceRequestOrder).Update(0x0, {{0x48, 0x52, 0x39, 0x69, 0x9c, 0x80, 0x1d, 0xf5, 0x8, ...}, ...}, ...)
    /go/pkg/mod/github.com/anacrolix/torrent@v1.47.0/request-strategy/piece-request-order.go:55 +0x55
github.com/anacrolix/torrent.(*Torrent).updatePieceRequestOrder(0xc02cded500, 0x3b3)
    /go/pkg/mod/github.com/anacrolix/torrent@v1.47.0/torrent-piece-request-order.go:11 +0xe9
github.com/anacrolix/torrent.(*Piece).pendChunkIndex(0xc03b255208, 0x1?)
    /go/pkg/mod/github.com/anacrolix/torrent@v1.47.0/piece.go:90 +0x57
github.com/anacrolix/torrent.(*Torrent).pendRequest(0x1?, 0x82c300?)
    /go/pkg/mod/github.com/anacrolix/torrent@v1.47.0/torrent.go:1275 +0x50
github.com/anacrolix/torrent.(*Peer).receiveChunk(0xc02959a900, 0xc03b336a80)
    /go/pkg/mod/github.com/anacrolix/torrent@v1.47.0/peerconn.go:1527 +0xb85
github.com/anacrolix/torrent.(*PeerConn).mainReadLoop(0xc02959a900)
    /go/pkg/mod/github.com/anacrolix/torrent@v1.47.0/peerconn.go:1255 +0x985
github.com/anacrolix/torrent.(*Client).runHandshookConn(0xc0009ee000, 0xc02959a900, 0xc02cded500)
    /go/pkg/mod/github.com/anacrolix/torrent@v1.47.0/client.go:994 +0x4a5
github.com/anacrolix/torrent.(*Torrent).logRunHandshookConn(0xc02cded500, 0xc02cded500?, 0x0, {0x14?})
    /go/pkg/mod/github.com/anacrolix/torrent@v1.47.0/torrent.go:1615 +0x52
github.com/anacrolix/torrent.(*Torrent).runHandshookConnLoggingErr(...)
    /go/pkg/mod/github.com/anacrolix/torrent@v1.47.0/torrent.go:1622
github.com/anacrolix/torrent.(*Client).outgoingConnection(0xc0009ee000, 0x443a25?, {0x1538980?, 0xc01ebd9be0}, {0x11d5e6a, 0x2}, 0x0)
    /go/pkg/mod/github.com/anacrolix/torrent@v1.47.0/client.go:797 +0x3bb
created by github.com/anacrolix/torrent.(*Torrent).initiateConn
    /go/pkg/mod/github.com/anacrolix/torrent@v1.47.0/torrent.go:2269 +0x25d
anacrolix commented 1 year ago

Could you try with v1.48.0, which includes https://github.com/anacrolix/torrent/commit/f75989863cfc4abfc14c1b4ab9a634437569ef27?

vintikzzz commented 1 year ago

This tag is not available, can you push it?

go get -u 'github.com/anacrolix/torrent@v1.48.0'
go: github.com/anacrolix/torrent@v1.48.0: invalid version: unknown revision v1.48.0

it looks like you just forgot leading 'v' letter

vintikzzz commented 1 year ago

can't build project after update

go build .
# github.com/anacrolix/dht/v2/k-nearest-nodes
../../../../go/pkg/mod/github.com/anacrolix/dht/v2@v2.19.2/k-nearest-nodes/k-nearest-nodes.go.go:21:9: cannot use generic type immutable.SortedMap[K comparable, V any] without instantiation
../../../../go/pkg/mod/github.com/anacrolix/dht/v2@v2.19.2/k-nearest-nodes/k-nearest-nodes.go.go:29:33: type lessComparer of lessComparer{…} does not match immutable.Comparer[K] (cannot infer K and V)
# github.com/anacrolix/dht/v2/containers
../../../../go/pkg/mod/github.com/anacrolix/dht/v2@v2.19.2/containers/addr-maybe-ids-by-distance.go:21:6: cannot use generic type stmutil.Settish[K stmutil.KeyConstraint] without instantiation
../../../../go/pkg/mod/github.com/anacrolix/dht/v2@v2.19.2/containers/addr-maybe-ids-by-distance.go:42:47: interface{} does not implement comparable
anacrolix commented 1 year ago

Thanks, I've pushed v1.48.0.

anacrolix commented 1 year ago

can't build project after update

go build .
# github.com/anacrolix/dht/v2/k-nearest-nodes
../../../../go/pkg/mod/github.com/anacrolix/dht/v2@v2.19.2/k-nearest-nodes/k-nearest-nodes.go.go:21:9: cannot use generic type immutable.SortedMap[K comparable, V any] without instantiation
../../../../go/pkg/mod/github.com/anacrolix/dht/v2@v2.19.2/k-nearest-nodes/k-nearest-nodes.go.go:29:33: type lessComparer of lessComparer{…} does not match immutable.Comparer[K] (cannot infer K and V)
# github.com/anacrolix/dht/v2/containers
../../../../go/pkg/mod/github.com/anacrolix/dht/v2@v2.19.2/containers/addr-maybe-ids-by-distance.go:21:6: cannot use generic type stmutil.Settish[K stmutil.KeyConstraint] without instantiation
../../../../go/pkg/mod/github.com/anacrolix/dht/v2@v2.19.2/containers/addr-maybe-ids-by-distance.go:42:47: interface{} does not implement comparable

Either go get github.com/anacrolix/dht/v2@master, or ensure that your indirect dependencies on github.com/anacrolix/stm and github.com/benbjohnson/immutable aren't ahead of those declared by github.com/anacrolix/dht/v2 and github.com/anacrolix/torrent. It's likely you've upgraded anacrolix/dht/v2 at some point, then downgraded again.

vintikzzz commented 1 year ago

Either go get github.com/anacrolix/dht/v2@master

Thanks for advice, it builds right now! We have to wait for a day to check.

vintikzzz commented 1 year ago

@anacrolix I have checked logs last day. The issue seems to be resolved! Great job! 👍

anacrolix commented 1 year ago

Thanks @vintikzzz

vintikzzz commented 1 year ago

Got the same today...

2023-01-18T16:35:20+0000 ERR [main client github.com/anacrolix/torrent.(*Peer).receiveChunk:1536]: *torrent.PeerConn 0xc01b011b00 [id="-qB4500-5ZtA84Bi4!X_", exts=0000000000180005, v="qBittorrent/4.5.0"]: writing received chunk 151128: short write
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x10 pc=0xc7a455]

goroutine 12790870 [running]:
github.com/anacrolix/torrent/request-strategy.(*PieceRequestOrder).Update(0x0, {{0x2e, 0xdf, 0x2d, 0xc5, 0x3d, 0xdd, 0xc1, 0x2b, 0xd0, ...}, ...}, ...)
        /go/pkg/mod/github.com/anacrolix/torrent@v1.48.0/request-strategy/piece-request-order.go:55 +0x55
github.com/anacrolix/torrent.(*Torrent).updatePieceRequestOrder(0xc036ac0000, 0x24e)
        /go/pkg/mod/github.com/anacrolix/torrent@v1.48.0/torrent-piece-request-order.go:11 +0xe9
github.com/anacrolix/torrent.(*Piece).pendChunkIndex(0xc036009fd0, 0x1?)
        /go/pkg/mod/github.com/anacrolix/torrent@v1.48.0/piece.go:91 +0x57
github.com/anacrolix/torrent.(*Torrent).pendRequest(0x1?, 0x152260?)
        /go/pkg/mod/github.com/anacrolix/torrent@v1.48.0/torrent.go:1275 +0x50
github.com/anacrolix/torrent.(*Peer).receiveChunk(0xc01b011b00, 0xc034d57c00)
        /go/pkg/mod/github.com/anacrolix/torrent@v1.48.0/peerconn.go:1537 +0xc05
github.com/anacrolix/torrent.(*PeerConn).mainReadLoop(0xc01b011b00)
        /go/pkg/mod/github.com/anacrolix/torrent@v1.48.0/peerconn.go:1260 +0x985
github.com/anacrolix/torrent.(*Client).runHandshookConn(0xc0006ca480, 0xc01b011b00, 0xc036ac0000)
        /go/pkg/mod/github.com/anacrolix/torrent@v1.48.0/client.go:990 +0x4a5
github.com/anacrolix/torrent.(*Torrent).logRunHandshookConn(0xc036ac0000, 0xc036ac0000?, 0x0, {0x13?})
        /go/pkg/mod/github.com/anacrolix/torrent@v1.48.0/torrent.go:1615 +0x52
github.com/anacrolix/torrent.(*Torrent).runHandshookConnLoggingErr(...)
        /go/pkg/mod/github.com/anacrolix/torrent@v1.48.0/torrent.go:1622
github.com/anacrolix/torrent.(*Client).outgoingConnection(0xc0006ca480, 0x443b65?, {0x1557440?, 0xc020a282e0}, {0x11f2b8a, 0x2}, 0x0)
        /go/pkg/mod/github.com/anacrolix/torrent@v1.48.0/client.go:793 +0x3bb
created by github.com/anacrolix/torrent.(*Torrent).initiateConn
        /go/pkg/mod/github.com/anacrolix/torrent@v1.48.0/torrent.go:2269 +0x25d

To be honest, it's first time after previous fix.

vintikzzz commented 1 year ago

@anacrolix Hi! This issue still persists.

anacrolix commented 1 year ago

I haven't had time to look into this. Do I recall correctly that you were setting very large chunk sizes in your project?

vintikzzz commented 1 year ago

@anacrolix no, settings are quite usual

cfg := torrent.NewDefaultClientConfig()
cfg.NoUpload = true
cfg.Seed = false
cfg.DefaultStorage = NewMMap(s.dataDir)
cfg.DisableUTP = true
cfg.DisableWebtorrent = true
cfg.EstablishedConnsPerTorrent = 100
cfg.HalfOpenConnsPerTorrent = 50
cfg.TorrentPeersHighWater = 1000
cfg.TorrentPeersLowWater = 100
cfg.TotalHalfOpenConns = 5000

I've already forked your project and done the fix. I have to do some trials during weekends. If everything will be fine, I will do a pull request on Monday.

vintikzzz commented 1 year ago

made a PR #807

anacrolix commented 1 year ago

Interesting, the PR got me thinking. Is it possible the torrent has been closed while the chunk is processed? It looks like that the torrent has been removed from the piece request order store in the Client.

vintikzzz commented 1 year ago

It is quite possible, I also thought about it. In my use case, I delete torrents from a client that were not used for a specific period of time.

anacrolix commented 1 year ago

I do the same thing, but it's interesting I haven't seen the bug. I think there's a better fix checking if it torrent is closed in receiveChunk.

anacrolix commented 1 year ago

The place the panic occurs is the only code path to place you fixed in your PR. So I think it's the best fix for now. Thank you!