Optimizations for 10MB+ wikis

[lambrospetrou]

Things to do:

• streams in request and response, without in-memory buffering outside the WikiDO.
• chunking/merging the wiki doc should be done in combination with streams to avoid breaking due to the 128MB memory limit.

Other things as followups (create separate issues)

• Use Workers KV or R2 for too big wikis. It's going to be slow anyway in those cases, so streaming the response from KV or R2 could be OK. The metadata should still be in the DO.

[lambrospetrou]

https://github.com/lambrospetrou/tiddlyflare/commit/a3a9d06b16c411063214948355230c8d4a801092

[lambrospetrou]

In https://github.com/lambrospetrou/tiddlyflare/commit/1f78357c76ee52838b82524675d1ae79ae318f49 there is streaming of the file back to the client by iterating over the SQLite cursor and writing into the ReadableStream returned in the response.

For files with less than 5 chunks, about 10MB, we assemble the whole file in memory and return that in a single chunk and also keep it in memory for faster serving of the next request.

[lambrospetrou]

We have streaming now in both sides, reading and writing the wiki files.

GZIP will remain for later as further optimization. Will track separately.