search

yorkie-team / yorkie

Yorkie is a document store for collaborative applications.
https://yorkie.dev
Apache License 2.0
782 stars 145 forks source link

Snapshot overflow #267

Open hackerwins opened 2 years ago

hackerwins commented 2 years ago

What happened:

If a document's snapshot exceeds 16 MB, the following error occurs:

2021-11-04T06:17:47.355Z    ERROR   an inserted document is too large
github.com/yorkie-team/yorkie/yorkie/backend/db/mongo.(*Client).CreateSnapshotInfo
    /app/yorkie/backend/db/mongo/client.go:371
github.com/yorkie-team/yorkie/yorkie/packs.storeSnapshot
    /app/yorkie/packs/packs.go:418
github.com/yorkie-team/yorkie/yorkie/packs.PushPull.func1
    /app/yorkie/packs/packs.go:150
github.com/yorkie-team/yorkie/yorkie/backend.(*Backend).AttachGoroutine.func1
    /app/yorkie/backend/backend.go:164
2021-11-04T06:17:47.366Z    ERROR   an inserted document is too large
github.com/yorkie-team/yorkie/yorkie/packs.PushPull.func1
    /app/yorkie/packs/packs.go:155
github.com/yorkie-team/yorkie/yorkie/backend.(*Backend).AttachGoroutine.func1
    /app/yorkie/backend/backend.go:164

This is because there is a limit on the size of documents in MongoDB.

To store documents larger than the maximum size, MongoDB provides the GridFS API. See mongofiles and the documentation for your driver for more information about GridFS.

It might be a good idea to check if we can use GridFS.

What you expected to happen:

How to reproduce it (as minimally and precisely as possible):

Anything else we need to know?:

Environment:

heka1024 commented 2 years ago

Could you assign this issue to me? cc. @dc7303

dc7303 commented 2 years ago

@heka1024 Sure. 😄