kappa-db / kappa-core

Minimal peer-to-peer database, based on kappa architecture.
254 stars 22 forks source link

pause/resume and manual feed/view mapping #6

Open ghost opened 5 years ago

ghost commented 5 years ago

With pause and resume methods on views, we can support some advanced sparse patterns and lazy calculation.

This could be done with:

core.use('cool', {
  api: { /* ... */ },
  map: function (rows, next) {
    // ...
    next()
  },
  pause: true
})
core.on('feed', function (feed) {
  if (feed.meta && feed.meta.type === 'cool') {
    core.resume('cool', feed.key)
})

Manually running views would be nice for compute-heavy workloads which might be cumbersome to run during/immediately after replication. And if you can manually add a log to a view later, you can do authentication checks first.

This feature would also allow us to have multi-log structures where views only apply to certain logs but not all logs for performance reasons. You could also have bulk logs which are referred to by other logs but the data in those bulk logs is not processed by a view directly. But that feature would also depend on https://github.com/kappa-db/multifeed/issues/2