Database-agnostic streaming replication

[JaneJeon]

If we want FTS via external document stores, then we need to pipe changes from the "truth store" to the search index.

While there are native ways to stream changes (such as zombodb for Postgres), 1. you often can't install plugins in DBaaS providers, 2. it doesn't work for different db's, 3. how would you even filter objects (e.g. private) and properties?

Using the bulk update option in #34, it might be possible to do this by running a repeating job that checks for last "updated_at" (inclusive, since we don’t care if we have to do it more than once; plus, this way we don’t lock the table), drags all objects and push them to the search index.

And we can avoid locking by keeping a "bookmark" thru job data (eliminating the need for transactions) instead.

https://gocardless.com/blog/syncing-postgres-to-elasticsearch-lessons-learned/

[JaneJeon]

Now there's a bulk add option with Bull Queue: https://github.com/OptimalBits/bull/commit/0d27fd0fdd5c3d868bf20dd9361147b6a38fc66c (undocumented)