anycable / anycable-go

AnyCable real-time server
https://anycable.io
MIT License
375 stars 65 forks source link

Panic during history retrieval with 1.4.2 fix #186

Closed yenshirak closed 1 year ago

yenshirak commented 1 year ago

Tell us about your environment

AnyCable-Go version: 1.4.2

AnyCable gem version: 1.4.1

We use the redisx broadcast adapter, the memory broker and the actioncable-v1-ext-json protocol, 1 anycable-go instance, no pub/sub. JWT authentication is enforced, we don't use RPC.

What did you do?

We upgraded to 1.4.2 and switched to the actioncable-v1-ext-json protocol.

What did you expect to happen?

No panics.

What actually happened?

The anycable-go service still panics but it seems less frequent:

August 09, 2023 at 10:44 (UTC+2:00) panic: runtime error: slice bounds out of range [11:1]  7bc68369745646b6a57f3159a8d59196    anycable
August 09, 2023 at 10:44 (UTC+2:00) goroutine 879601 [running]: 7bc68369745646b6a57f3159a8d59196    anycable
August 09, 2023 at 10:44 (UTC+2:00) github.com/anycable/anycable-go/broker.(*memstream).filterByOffset(0xc000b09180, 0xc0008cb6b0?, 0xc000799c58)   7bc68369745646b6a57f3159a8d59196    anycable
August 09, 2023 at 10:44 (UTC+2:00) /home/runner/work/anycable-go/anycable-go/broker/memory.go:104 +0x285   7bc68369745646b6a57f3159a8d59196    anycable
August 09, 2023 at 10:44 (UTC+2:00) github.com/anycable/anycable-go/broker.(*Memory).HistoryFrom(0xc0000d9440, {0xc0008cb6b0, 0x25}, {0xc0003e056c, 0x4}, 0x0?) 7bc68369745646b6a57f3159a8d59196    anycable
August 09, 2023 at 10:44 (UTC+2:00) /home/runner/work/anycable-go/anycable-go/broker/memory.go:239 +0x1c5   7bc68369745646b6a57f3159a8d59196    anycable
August 09, 2023 at 10:44 (UTC+2:00) github.com/anycable/anycable-go/node.(*Node).retreiveHistory(0xc0001dc700, 0xc000799da0, {0xc0002755d0?, 0x1, 0x799d70?})   7bc68369745646b6a57f3159a8d59196    anycable
August 09, 2023 at 10:44 (UTC+2:00) /home/runner/work/anycable-go/anycable-go/node/node.go:540 +0x229   7bc68369745646b6a57f3159a8d59196    anycable
August 09, 2023 at 10:44 (UTC+2:00) github.com/anycable/anycable-go/node.(*Node).History(0xc0000d9440?, 0xc000000d80, 0xc0007ef1c0) 7bc68369745646b6a57f3159a8d59196    anycable
August 09, 2023 at 10:44 (UTC+2:00) /home/runner/work/anycable-go/anycable-go/node/node.go:509 +0x213   7bc68369745646b6a57f3159a8d59196    anycable
August 09, 2023 at 10:44 (UTC+2:00) github.com/anycable/anycable-go/node.(*Node).Subscribe(0xc0001dc700, 0xc000000d80, 0xc0007ef1c0)    7bc68369745646b6a57f3159a8d59196    anycable
August 09, 2023 at 10:44 (UTC+2:00) /home/runner/work/anycable-go/anycable-go/node/node.go:403 +0x449   7bc68369745646b6a57f3159a8d59196    anycable
August 09, 2023 at 10:44 (UTC+2:00) github.com/anycable/anycable-go/node.(*Node).HandleCommand(0xe3ffd7?, 0xc000000d80, 0xc0007ef1c0)   7bc68369745646b6a57f3159a8d59196    anycable
August 09, 2023 at 10:44 (UTC+2:00) /home/runner/work/anycable-go/anycable-go/node/node.go:131 +0x111   7bc68369745646b6a57f3159a8d59196    anycable
August 09, 2023 at 10:44 (UTC+2:00) github.com/anycable/anycable-go/node.(*Session).ReadMessage(0xc000000d80, {0xc00033e000, 0x150, 0x200}) 7bc68369745646b6a57f3159a8d59196    anycable
August 09, 2023 at 10:44 (UTC+2:00) /home/runner/work/anycable-go/anycable-go/node/session.go:303 +0xde 7bc68369745646b6a57f3159a8d59196    anycable
August 09, 2023 at 10:44 (UTC+2:00) github.com/anycable/anycable-go/node.(*Session).Serve.func1()   7bc68369745646b6a57f3159a8d59196    anycable
August 09, 2023 at 10:44 (UTC+2:00) /home/runner/work/anycable-go/anycable-go/node/session.go:255 +0x6c 7bc68369745646b6a57f3159a8d59196    anycable
August 09, 2023 at 10:44 (UTC+2:00) created by github.com/anycable/anycable-go/node.(*Session).Serve    7bc68369745646b6a57f3159a8d59196    anycable
August 09, 2023 at 10:44 (UTC+2:00) /home/runner/work/anycable-go/anycable-go/node/session.go:238 +0x72 7bc68369745646b6a57f3159a8d59196    anycable

Please let me know if you need any additional info or help with debugging and fixing the issue.

palkan commented 1 year ago

Thanks for reporting! I was finally able to reproduce it locally. Will ship a fix soon.

palkan commented 1 year ago

Okay, fixed here https://github.com/anycable/anycable-go/commit/e8fd29c217e813d7fb367f6d7bf11864f3954f8a. Available in the edge image today, or in the upcoming release (~Thu-Fri).

yenshirak commented 1 year ago

Great, thank you!