edwargix
commented
2 years ago Hi, I am experiencing an issue that looks to be the same as what you're facing:
time="2021-12-31T20:10:05Z" level=fatal msg="error syncing: ProcessResponse panicked! since= panic=runtime error: slice bounds out of range [:160] with capacity 152\ngoroutine 1 [running]:\nruntime/debug.Stack()\n\t/usr/lib/go/src/runtime/debug/stack.go:24 +0x65\nmaunium.net/go/mautrix.(*DefaultSyncer).ProcessResponse.func1()\n\t/root/go/pkg/mod/maunium.net/go/mautrix@v0.10.6/sync.go:133 +0x66\npanic({0x8ba520, 0xc000400690})\n\t/usr/lib/go/src/runtime/panic.go:1038 +0x215\nmaunium.net/go/mautrix/crypto/canonicaljson.RawJSONFromResult(...)\n\t/root/go/pkg/mod/maunium.net/go/mautrix@v0.10.6/crypto/canonicaljson/json.go:274\nmaunium.net/go/mautrix/crypto/canonicaljson.sortJSONObject.func1({0x3, {0xc000356869, 0x17}, {0xc00035686a, 0x15}, 0x0, 0x89, {0x0, 0x0, 0x0}}, ...)\n\t/root/go/pkg/mod/maunium.net/go/mautrix@v0.10.6/crypto/canonicaljson/json.go:113 +0x2d4\ngithub.com/tidwall/gjson.Result.ForEach({0x5, {0xc000356868, 0x8a}, {0x0, 0x0}, 0x0, 0x88, {0x0, 0x0, 0x0}}, ...)\n\t/root/go/pkg/mod/github.com/tidwall/gjson@v1.12.1/gjson.go:293 +0x52c\nmaunium.net/go/mautrix/crypto/canonicaljson.sortJSONObject({0x5, {0xc000356868, 0x8a}, {0x0, 0x0}, 0x0, 0x88, {0x0, 0x0, 0x0}}, ...)\n\t/root/go/pkg/mod/maunium.net/go/mautrix@v0.10.6/crypto/canonicaljson/json.go:110 +0x165\nmaunium.net/go/mautrix/crypto/canonicaljson.sortJSONValue({0x5, {0xc000356868, 0x8a}, {0x0, 0x0}, 0x0, 0x88, {0x0, 0x0, 0x0}}, ...)\n\t/root/go/pkg/mod/maunium.net/go/mautrix@v0.10.6/crypto/canonicaljson/json.go:62 +0x289\nmaunium.net/go/mautrix/crypto/canonicaljson.sortJSONObject({0x5, {0xc0003567e0, 0x113}, {0x0, 0x0}, 0x0, 0x0, {0x0, 0x0, 0x0}}, ...)\n\t/root/go/pkg/mod/maunium.net/go/mautrix@v0.10.6/crypto/canonicaljson/json.go:136 +0x5f3\nmaunium.net/go/mautrix/crypto/canonicaljson.sortJSONValue({0x5, {0xc0003567e0, 0x113}, {0x0, 0x0}, 0x0, 0x0, {0x0, 0x0, 0x0}}, ...)\n\t/root/go/pkg/mod/maunium.net/go/mautrix@v0.10.6/crypto/canonicaljson/json.go:62 +0x289\nmaunium.net/go/mautrix/crypto/canonicaljson.SortJSON({0xc000356360, 0xc000258f00, 0x113}, {0xc0003565a0, 0x0, 0x113})\n\t/root/go/pkg/mod/maunium.net/go/mautrix@v0.10.6/crypto/canonicaljson/json.go:51 +0x23a\nmaunium.net/go/mautrix/crypto/canonicaljson.CanonicalJSONAssumeValid({0xc000356120, 0x113, 0x120})\n\t/root/go/pkg/mod/maunium.net/go/mautrix@v0.10.6/crypto/canonicaljson/json.go:42 +0x96\nmaunium.net/go/mautrix/crypto/olm.(*Utility).VerifySignatureJSON(0x50, {0x8bc0e0, 0xc0003920f0}, {0xc000400408, 0x15}, {0xc0004140f0, 0xa}, {0xc000424210, 0x2b})\n\t/root/go/pkg/mod/maunium.net/go/mautrix@v0.10.6/crypto/olm/utility.go:146 +0x2c7\nmaunium.net/go/mautrix/crypto/olm.VerifySignatureJSON({0x8bc0e0, 0xc0003920f0}, {0xc000400408, 0x15}, {0xc0004140f0, 0xa}, {0xc000424210, 0x2b})\n\t/root/go/pkg/mod/maunium.net/go/mautrix@v0.10.6/crypto/olm/utility.go:159 +0xe5\nmaunium.net/go/mautrix/crypto.(*OlmMachine).validateDevice(0x0, {0xc000400408, 0x2}, {0xc0004140f0, 0xa}, {{0xc0004003c0, 0x15}, {0xc0004140d0, 0xa}, {0xc0001ae880, ...}, ...}, ...)\n\t/root/go/pkg/mod/maunium.net/go/mautrix@v0.10.6/crypto/devicelist.go:165 +0x21d\nmaunium.net/go/mautrix/crypto.(*OlmMachine).fetchKeys(0xc0000fc300, {0xc0003dc520, 0x1, 0x1}, {0x0, 0x0}, 0x1)\n\t/root/go/pkg/mod/maunium.net/go/mautrix@v0.10.6/crypto/devicelist.go:75 +0xf2c\nmaunium.net/go/mautrix/crypto.(*OlmMachine).GetOrFetchDevice(0xc0000fc300, {0xc000437a10, 0x2b}, {0xc0003e03a0, 0xa})\n\t/root/go/pkg/mod/maunium.net/go/mautrix@v0.10.6/crypto/machine.go:343 +0x108\nmaunium.net/go/mautrix/crypto.(*OlmMachine).DecryptMegolmEvent(0xc0000fc300, 0xc000202780)\n\t/root/go/pkg/mod/maunium.net/go/mautrix@v0.10.6/crypto/decryptmegolm.go:58 +0x457\nmain.main.func3(0x878d60, 0xc000202780)\n\t/main.go:70 +0x54\nmaunium.net/go/mautrix.(*DefaultSyncer).notifyListeners(0xc0000667d0, 0xc000296510, 0xc000202780)\n\t/root/go/pkg/mod/maunium.net/go/mautrix@v0.10.6/sync.go:203 +0xd7\nmaunium.net/go/mautrix.(*DefaultSyncer).processSyncEvent(0xc0000667d0, {0xc0000193b0, 0xc00018fcb0}, 0xc000202780, 0x22)\n\t/root/go/pkg/mod/maunium.net/go/mautrix@v0.10.6/sync.go:193 +0x105\nmaunium.net/go/mautrix.(*DefaultSyncer).processSyncEvents(0xc000159ab8, {0xc0000193b0, 0x21}, {0xc00017e200, 0x32, 0xc00010e540}, 0xc000108d00)\n\t/root/go/pkg/mod/maunium.net/go/mautrix@v0.10.6/sync.go:164 +0x4a\nmaunium.net/go/mautrix.(*DefaultSyncer).ProcessResponse(0xc0000667d0, 0xc0000ce000, {0x0, 0x0})\n\t/root/go/pkg/mod/maunium.net/go/mautrix@v0.10.6/sync.go:148 +0x293\nmaunium.net/go/mautrix.(*Client).SyncWithContext(0xc0000c82a0, {0x96c470, 0xc00001c038})\n\t/root/go/pkg/mod/maunium.net/go/mautrix@v0.10.6/client.go:284 +0x46f\nmaunium.net/go/mautrix.(*Client).Sync(...)\n\t/root/go/pkg/mod/maunium.net/go/mautrix@v0.10.6/client.go:225\nmain.main()\n\t/main.go:114 +0x10f9\n"Here's the log message with actual newlines and tabs:
time="2021-12-31T20:10:05Z" level=fatal msg="error syncing: ProcessResponse panicked! since= panic=runtime error: slice bounds out of range [:160] with capacity 152
goroutine 1 [running]:
runtime/debug.Stack()
/usr/lib/go/src/runtime/debug/stack.go:24 +0x65
maunium.net/go/mautrix.(*DefaultSyncer).ProcessResponse.func1()
/root/go/pkg/mod/maunium.net/go/mautrix@v0.10.6/sync.go:133 +0x66
panic({0x8ba520, 0xc000400690})
/usr/lib/go/src/runtime/panic.go:1038 +0x215
maunium.net/go/mautrix/crypto/canonicaljson.RawJSONFromResult(...)
/root/go/pkg/mod/maunium.net/go/mautrix@v0.10.6/crypto/canonicaljson/json.go:274
maunium.net/go/mautrix/crypto/canonicaljson.sortJSONObject.func1({0x3, {0xc000356869, 0x17}, {0xc00035686a, 0x15}, 0x0, 0x89, {0x0, 0x0, 0x0}}, ...)
/root/go/pkg/mod/maunium.net/go/mautrix@v0.10.6/crypto/canonicaljson/json.go:113 +0x2d4
github.com/tidwall/gjson.Result.ForEach({0x5, {0xc000356868, 0x8a}, {0x0, 0x0}, 0x0, 0x88, {0x0, 0x0, 0x0}}, ...)
/root/go/pkg/mod/github.com/tidwall/gjson@v1.12.1/gjson.go:293 +0x52c
maunium.net/go/mautrix/crypto/canonicaljson.sortJSONObject({0x5, {0xc000356868, 0x8a}, {0x0, 0x0}, 0x0, 0x88, {0x0, 0x0, 0x0}}, ...)
/root/go/pkg/mod/maunium.net/go/mautrix@v0.10.6/crypto/canonicaljson/json.go:110 +0x165
maunium.net/go/mautrix/crypto/canonicaljson.sortJSONValue({0x5, {0xc000356868, 0x8a}, {0x0, 0x0}, 0x0, 0x88, {0x0, 0x0, 0x0}}, ...)
/root/go/pkg/mod/maunium.net/go/mautrix@v0.10.6/crypto/canonicaljson/json.go:62 +0x289
maunium.net/go/mautrix/crypto/canonicaljson.sortJSONObject({0x5, {0xc0003567e0, 0x113}, {0x0, 0x0}, 0x0, 0x0, {0x0, 0x0, 0x0}}, ...)
/root/go/pkg/mod/maunium.net/go/mautrix@v0.10.6/crypto/canonicaljson/json.go:136 +0x5f3
maunium.net/go/mautrix/crypto/canonicaljson.sortJSONValue({0x5, {0xc0003567e0, 0x113}, {0x0, 0x0}, 0x0, 0x0, {0x0, 0x0, 0x0}}, ...)
/root/go/pkg/mod/maunium.net/go/mautrix@v0.10.6/crypto/canonicaljson/json.go:62 +0x289
maunium.net/go/mautrix/crypto/canonicaljson.SortJSON({0xc000356360, 0xc000258f00, 0x113}, {0xc0003565a0, 0x0, 0x113})
/root/go/pkg/mod/maunium.net/go/mautrix@v0.10.6/crypto/canonicaljson/json.go:51 +0x23a
maunium.net/go/mautrix/crypto/canonicaljson.CanonicalJSONAssumeValid({0xc000356120, 0x113, 0x120})
/root/go/pkg/mod/maunium.net/go/mautrix@v0.10.6/crypto/canonicaljson/json.go:42 +0x96
maunium.net/go/mautrix/crypto/olm.(*Utility).VerifySignatureJSON(0x50, {0x8bc0e0, 0xc0003920f0}, {0xc000400408, 0x15}, {0xc0004140f0, 0xa}, {0xc000424210, 0x2b})
/root/go/pkg/mod/maunium.net/go/mautrix@v0.10.6/crypto/olm/utility.go:146 +0x2c7
maunium.net/go/mautrix/crypto/olm.VerifySignatureJSON({0x8bc0e0, 0xc0003920f0}, {0xc000400408, 0x15}, {0xc0004140f0, 0xa}, {0xc000424210, 0x2b})
/root/go/pkg/mod/maunium.net/go/mautrix@v0.10.6/crypto/olm/utility.go:159 +0xe5
maunium.net/go/mautrix/crypto.(*OlmMachine).validateDevice(0x0, {0xc000400408, 0x2}, {0xc0004140f0, 0xa}, {{0xc0004003c0, 0x15}, {0xc0004140d0, 0xa}, {0xc0001ae880, ...}, ...}, ...)
/root/go/pkg/mod/maunium.net/go/mautrix@v0.10.6/crypto/devicelist.go:165 +0x21d
maunium.net/go/mautrix/crypto.(*OlmMachine).fetchKeys(0xc0000fc300, {0xc0003dc520, 0x1, 0x1}, {0x0, 0x0}, 0x1)
/root/go/pkg/mod/maunium.net/go/mautrix@v0.10.6/crypto/devicelist.go:75 +0xf2c
maunium.net/go/mautrix/crypto.(*OlmMachine).GetOrFetchDevice(0xc0000fc300, {0xc000437a10, 0x2b}, {0xc0003e03a0, 0xa})
/root/go/pkg/mod/maunium.net/go/mautrix@v0.10.6/crypto/machine.go:343 +0x108
maunium.net/go/mautrix/crypto.(*OlmMachine).DecryptMegolmEvent(0xc0000fc300, 0xc000202780)
/root/go/pkg/mod/maunium.net/go/mautrix@v0.10.6/crypto/decryptmegolm.go:58 +0x457
main.main.func3(0x878d60, 0xc000202780)
/main.go:70 +0x54
maunium.net/go/mautrix.(*DefaultSyncer).notifyListeners(0xc0000667d0, 0xc000296510, 0xc000202780)
/root/go/pkg/mod/maunium.net/go/mautrix@v0.10.6/sync.go:203 +0xd7
maunium.net/go/mautrix.(*DefaultSyncer).processSyncEvent(0xc0000667d0, {0xc0000193b0, 0xc00018fcb0}, 0xc000202780, 0x22)
/root/go/pkg/mod/maunium.net/go/mautrix@v0.10.6/sync.go:193 +0x105
maunium.net/go/mautrix.(*DefaultSyncer).processSyncEvents(0xc000159ab8, {0xc0000193b0, 0x21}, {0xc00017e200, 0x32, 0xc00010e540}, 0xc000108d00)
/root/go/pkg/mod/maunium.net/go/mautrix@v0.10.6/sync.go:164 +0x4a
maunium.net/go/mautrix.(*DefaultSyncer).ProcessResponse(0xc0000667d0, 0xc0000ce000, {0x0, 0x0})
/root/go/pkg/mod/maunium.net/go/mautrix@v0.10.6/sync.go:148 +0x293
maunium.net/go/mautrix.(*Client).SyncWithContext(0xc0000c82a0, {0x96c470, 0xc00001c038})
/root/go/pkg/mod/maunium.net/go/mautrix@v0.10.6/client.go:284 +0x46f
maunium.net/go/mautrix.(*Client).Sync(...)
/root/go/pkg/mod/maunium.net/go/mautrix@v0.10.6/client.go:225
main.main()
/main.go:114 +0x10f9
"I've found that the following patch fixes the issue, though at the expense of removing an optimization for gjson:
From f4965a653a70e66786665431442c607b54de73e4 Mon Sep 17 00:00:00 2001
From: David Florness <david@florness.com>
Date: Fri, 31 Dec 2021 14:19:28 -0600
Subject: [PATCH] Don't use gjson slice optimization
This is a workaround for https://github.com/mautrix/go/issues/52
---
crypto/canonicaljson/json.go | 10 +---------
1 file changed, 1 insertion(+), 9 deletions(-)
diff --git a/crypto/canonicaljson/json.go b/crypto/canonicaljson/json.go
index 8c12176..231d798 100644
--- a/crypto/canonicaljson/json.go
+++ b/crypto/canonicaljson/json.go
@@ -266,15 +266,7 @@ func readHexDigits(input []byte) uint32 {
// RawJSONFromResult extracts the raw JSON bytes pointed to by result.
// input must be the json bytes that were used to generate result
func RawJSONFromResult(result gjson.Result, input []byte) (RawJSON []byte) {
- // This is lifted from gjson README. Basically, result.Raw is a copy of
- // the bytes we want, but its more efficient to take a slice.
- // If Index is 0 then for some reason we can't extract it from the original
- // JSON bytes.
- if result.Index > 0 {
- RawJSON = input[result.Index : result.Index+len(result.Raw)]
- } else {
- RawJSON = []byte(result.Raw)
- }
+ RawJSON = []byte(result.Raw)
return
}
--
2.32.0
I unfortunately haven't yet been able to find a proper fix, but hopefully this can be used as a workaround in the short term?
All the best.
rakshazi
tulir
Hello,
I'm trying to develop a bot on top of mautrix-go and faced odd issue - OlmMachine.ProcessSyncResponse panics on
canonicaljson.RawJSONFromResult. The issue reproduced on synapse 1.49.1, 1.49.2 and different matrix accounts (I created a new account just to be sure the issue is not caused by some kind of "state" saved into that account) and different mautrix-go versions (0.10.5, 0.10.7, 0.10.8).here log (level=trace) with stacktrace:
Please, help me to fix that