search

whyrusleeping / cbor-gen

Codegen for cbor codecs on your types
MIT License
35 stars 25 forks source link

Slice bounds out of range + panic in ReadStringWithMax #99

Closed lxcode closed 5 months ago

lxcode commented 5 months ago

While reading a repo with Indigo, I'm getting occasional panics like this:

time="2024-06-03T12:29:34Z" level=info msg="Connecting to WebSocket at: wss://bsky.network/xrpc/com.atproto.sync.subscribeRepos?cursor=552650495"
panic: runtime error: slice bounds out of range [:11974] with capacity 8192

goroutine 111 [running]:
github.com/whyrusleeping/cbor-gen.ReadStringWithMax({0x144cd80, 0x8706f9350}, 0xf4240)
        /home/det/go/pkg/mod/github.com/whyrusleeping/cbor-gen@v0.1.1/utils.go:389 +0x2d7
github.com/bluesky-social/indigo/api/bsky.(*EmbedExternal_External).UnmarshalCBOR(0x8705d24c0, {0x144cd80, 0x8706f9350})
        /home/det/go/pkg/mod/github.com/bluesky-social/indigo@v0.0.0-20240529174714-c1b16d4592a8/api/bsky/cbor_gen.go:1737 +0x565
github.com/bluesky-social/indigo/api/bsky.(*EmbedExternal).UnmarshalCBOR(0x8708ccc18, {0x144ca40, 0x8706f9320?})
        /home/det/go/pkg/mod/github.com/bluesky-social/indigo@v0.0.0-20240529174714-c1b16d4592a8/api/bsky/cbor_gen.go:1531 +0x499
github.com/bluesky-social/indigo/api/bsky.(*FeedPost_Embed).UnmarshalCBOR(0x8708f0b20, {0x144cd80?, 0x8706f9290?})
        /home/det/go/pkg/mod/github.com/bluesky-social/indigo@v0.0.0-20240529174714-c1b16d4592a8/api/bsky/feedpost.go:125 +0x1e5
github.com/bluesky-social/indigo/api/bsky.(*FeedPost).UnmarshalCBOR(0x87057cbb0, {0x144ca40, 0x8706f9260})
        /home/det/go/pkg/mod/github.com/bluesky-social/indigo@v0.0.0-20240529174714-c1b16d4592a8/api/bsky/cbor_gen.go:432 +0x6fc
github.com/bluesky-social/indigo/lex/util.CborDecodeValue({0x870884024, 0x3075, 0x3075})
        /home/det/go/pkg/mod/github.com/bluesky-social/indigo@v0.0.0-20240529174714-c1b16d4592a8/lex/util/decoder.go:89 +0x1e2
github.com/bluesky-social/indigo/repo.(*Repo).GetRecord(0x8706fa1e0, {0x145a5f8, 0x1e66e60}, {0x87050b320, 0x20})
        /home/det/go/pkg/mod/github.com/bluesky-social/indigo@v0.0.0-20240529174714-c1b16d4592a8/repo/repo.go:370 +0x14f
github.com/stanfordio/skyfall/pkg/stream.(*Stream).HandleRepoCommit(0x8708f0800, {0x145a5f8, 0x1e66e60}, 0x8700db970)
        /home/det/git/skyfall/pkg/stream/stream.go:109 +0x7e5
github.com/stanfordio/skyfall/pkg/stream.(*Stream).HandleStreamEvent(0x40c172?, {0x145a5f8?, 0x1e66e60?}, 0x472901?)
        /home/det/git/skyfall/pkg/stream/stream.go:81 +0xa6
github.com/bluesky-social/indigo/events/schedulers/autoscaling.(*Scheduler).worker(0x870000300)
        /home/det/go/pkg/mod/github.com/bluesky-social/indigo@v0.0.0-20240529174714-c1b16d4592a8/events/schedulers/autoscaling/autoscaling.go:214 +0x20c
created by github.com/bluesky-social/indigo/events/schedulers/autoscaling.NewScheduler in goroutine 109
        /home/det/go/pkg/mod/github.com/bluesky-social/indigo@v0.0.0-20240529174714-c1b16d4592a8/events/schedulers/autoscaling/autoscaling.go:105 +0x55e

I think what's going in is:

In a case where readString is used, cbor-gen callsreadStringWithMax with a max of MaxLength (8192), which matches the size of the slice returned by stringBufPool. However, Indigo is calling readStringWithMax directly, with a size of 1000000 in this case, larger than what stringBufPool returns. Not sure what the desired fix is — changing stringBufPool to take its own size parameter?

lxcode commented 5 months ago

I see now that this is under discussion at: https://github.com/whyrusleeping/cbor-gen/pull/95

Stebalien commented 5 months ago

Thanks for the reminder, I've merged that and released v0.1.2.