precog / platform

Advanced Analytics Engine for NoSQL Data
http://www.slamdata.com
GNU Affero General Public License v3.0
401 stars 64 forks source link

Remove MongoDB dependency #523

Open jdegoes opened 11 years ago

jdegoes commented 11 years ago

Precog currently uses MongoDB to persist metadata, such as account information, security information, and so forth. MongoDB is a database in its own right, which has to be installed and maintained. This takes a lot of time and domain knowledge.

In order to minimize the moving pieces in Precog, remove a point of failure, and simplify both initial installation of Precog and ongoing maintenance, the dependency on MongoDB needs to be completely eliminated.

We need to swap out MongoDB with a robust, pure Java embedded database (such as H2), which will make it easier to allow users to introduce redundancy in the future (by switching to a distributed SQL database).

The main criteria is that whatever we use to replace MongoDB should not require installation or maintenance of any kind; it needs to be idiot-proof, crash-proof (auto-recovery), and configuration-free.

Per @nuttycom, the main touch point in the code for the security service is ApiKeyFinder (what about for the accounts service?). The trickier bit will be refactoring and dead code elimination in the dvergr service.

nuttycom commented 11 years ago

This can be achieved by implementing the ApiKeyFinder trait to work directly atop an HSQLDB (or similar) database instance. The trickier bit here will be the refactoring and dead code elimination in the dvergr service (which should be undertaken irrespective of whether or not Mongo is no longer supported for account management.

Also: the auth and accounts services need to be merged, and their interfaces simplified.

On Sat, Oct 5, 2013 at 1:57 PM, John A. De Goes notifications@github.comwrote:

Precog currently uses MongoDB to persist metadata, such as account information, security information, and so forth. MongoDB is a database in its own right, which has to be installed and maintained. This takes a lot of time and domain knowledge.

In order to minimize the moving pieces in Precog, remove a point of failure, and simplify both initial installation of Precog and ongoing maintenance, the dependency on MongoDB needs to be completely eliminated.

We could swap out MongoDB with a robust, pure Java embedded database (such as H2), which would make it easier to allow users to introduce redundancy in the future.

The main criteria is that whatever we use to replace MongoDB should not require installation or maintenance of any kind; it needs to be idiot-proof, crash-proof (auto-recovery), and configuration-free.

— Reply to this email directly or view it on GitHubhttps://github.com/precog/platform/issues/523 .

jdegoes commented 11 years ago

Description updated with @nuttyom's feedback and #530 created for merging auth / accounts.

traycho-zz commented 10 years ago

Do you plan to implement this in near future or the repo won't be updated with new code ?

jdegoes commented 10 years ago

The repo will be updated with the new code as soon as it's ready.