search

neumino / rethinkdbdash

An advanced Node.js driver for RethinkDB with a connection pool, support for streams etc.
MIT License
848 stars 109 forks source link

High memory usage when using streams + changefeeds #357

Open marshall007 opened 6 years ago

marshall007 commented 6 years ago

We have a CLI task for keeping a redis cache populated from RethinkDB. It has a --watch argument which, when true, appends .changes({ includeInitial: true })('new_val') to each query used to rebuild the cache.

The code and queries being executed are otherwise identical, but running with --watch results in ~3x the memory consumption.

./bin/cache          130MB
./bin/cache --watch  330MB

I think this may be related to #263... perhaps the fix mentioned there (https://github.com/neumino/rethinkdbdash/commit/c323d4958383be41e16f525cc5cac061a4c9ef29) was not ported over to the stream handling code?

neumino commented 6 years ago

Do you have a script where this happen? Does cache just has a single query with changes({ includeInitial: true })?