earthstar-project / earthstar

Storage for private, distributed, offline-first applications.
https://earthstar-project.org
GNU Lesser General Public License v3.0
634 stars 20 forks source link

Specify max size limit for Earthstar documents #89

Closed cinnamon-bun closed 1 year ago

cinnamon-bun commented 3 years ago

The specification should say how large an Earthstar document is allowed to be. It's not specified right now!

More precisely, how long is the content field allowed to be.

Scuttlebutt's message limit is something like 8 kb. I think we could have something much larger, maybe 3 to 5 mb, because:

Let's choose a size that's useful for most individual files in a web context (javascript, images, etc) so people don't have to mess around with breaking files into chunks.

We already have a function stringLengthInBytes which counts the number of bytes needed to store a string after encoding it to UTF-8: https://github.com/earthstar-project/earthstar/blob/a19ff805b2739b423fc75edc178eb2dec58ff488/src/storage/query.ts#L186-L187

cinnamon-bun commented 3 years ago

Decided to set it to 4 million bytes (4,000,000 bytes) measured as utf-8 encoded bytes.