process isolation

[joeblew99]

This looks pretty interesting, and thanks for putting this out there...

I am using Dart for Desktop and Mobile, and will try to use semblast for my prototype, but wanted to ask a few things about process...

1. On mobile, have your tried to run this with Flutter ? On mobile i have tried two architecture:
   • running the DD as a Service, witht he UI running in a webview (and Angular2/Dart)
   • running everything in the same procces with Flutter.

So, it begs the question, what have you tried.

2. On Desktop, i use Electron / Dart. It shoudl be easy to use semblast running in a different process that the Electron runtime kicks off. But it would mean that i need to use semblast with a Dart based Http Service. Have you tried using Semblast with a Http Service fronting it ?

Lastly, are you thinking about extending semblast so that it can also be durable, in that it use the disk also ?

thanks in advance

[alextekartik]

On mobile, have your tried to run this with Flutter ?

Yes I have tried sembast successfully on flutter (about 2 months ago)

Have you tried using Semblast with a Http Service fronting it ?

Yes i have also tried sembast succesfully with a simple shelf rpc server.

Lastly, are you thinking about extending semblast so that it can also be durable, in that it use the disk also ?

Actually it is durable and it does write to the disk (if you use sembast_io.dart) on write (if you run test/io_factorytest.dart, you will see files generated in test/test_out). The main limitation is that it's consistency is only valid for single process and its whole content is in memory (ouch). In its behavior, features, it is very similar to NeDB for node.js (https://github.com/louischatriot/nedb). I have not done limitation test with millions of record, but besides being slow on startup (loading everything in memory), it should be pretty fast afterwards.

I would love to hear about Electron experiments (that I have not tried my self yet) and would be happy to hear feedback on improvements. If you can ensure that a single process is accessing the db, you could stick with sembast in the same process (which was actually its goal) as it would simplify coding a lot before extracting in a http service (here the benefits would be to be able to switch easily to a different database system without affecting your app)

[joeblew99]

thanks for the info...

Great that it works with flutter. I will give it a go for sure..

There is a good Electron Dart library here. It work really well and the dev is keen for people to dog food it and feedback... https://github.com/electron-dart

But i am not sure it will work with electron / dart. Everything is compiled to JS, so anything running in the Main Process (as opposed to the Render Process) is just JS that runs on top of nodeJS. Semblast will not run on top of NodeJS i suspect ? Not sure...

Someone of Slack told me today that google have an internal project to get the JS from Dart working with Nodejs though... I dont knwo more than that though at this stage.. But if it did work then as a developer i coudl use IPC to communicate between the backend (db with some logic) and the frontend (electron Render process). Using IPC would be preferred...

But the other way is to run it as a dart process (snap shot) i guess. this is why i suggested fronting it with a Dart http server.

We shoudl try it !!

[joeblew99]

hope you dont mind but i was asking about this and one of the flutter devs said that it a good idea to have a flutter db. so i made a feature request and listed this repo.

here is my feature request: https://github.com/flutter/flutter/issues/6676

[alextekartik]

Not a problem. I have a long term plan to have sembast work over node.js (I guess it needs some io bridging - sass dart is doing that). Regarding flutter, sembast was more to be a portable simple solution rather than a full robust (leveldb, sqlite) well known database implementation that flutter should provide at some point. I think it can fullfill the need until your feature requet gets a solution

[joeblew99]

great - glad i did not overstep..

I think that what you have now is fine for me at least. Originally i thought i needed to run the DB on top of Nodejs, but i realised i dont and can just run the DB inside the browser. Are you using local storage ?

I am using your DB to only store events over time (like a WAL - write ahead log). The events are coming downstream from the GUI Views as Actions (aka Flux pattern), and from the Server (aka CQRS pattern). So its basically an Event Store. Flux layer subscribes to the Store events (based on ordered time stamps), runs the transformations on the immutable Flux store data, and creates Materials Views / Projections that the GUI Views are bound to.

Both the Flutter, Web & Desktop app then are using identical code.

If you have time, let me know if this makes sense, and if you happen to see some lurking dragons..

Hope that makes sense.

[alextekartik]

sembast in the browser only runs for now in memory so you won't get persistence in the browser. 2 alternatives:

• use idb_shim which provides an idb like interface that can run on the browser (on top of idb) or in io (on top of sembast). That currently works and provides persistence for both browser and io (flutter & desktop).
• I actually have another fs_shim package (not updated for the latest strong mode yet though) which provides a file api on top on indexeddb. I could update sembast to use it instead of its own file abstraction

[joeblew99]

thanks for the two alternatives.

I checked out idb_shim, and it looks pretty good. I think its very useful addition to Semblast, especially with its factory pattern. Even despite the fact that not all browsers have great indexdb support( but i hear that safari and ios are getting better). For me, its not really an issue because i run the Web GUi inside Electron, and so am really just running on top of Chrome.

I am planning to match up Semblast with a Dart library that does facet search. At the moment it has no persistence client side. Its not my library, but really nice. I am a sicker for facet search. Its called Ducene (play on words of Lucene) https://bitbucket.org/ouava/ducene/overview

The data is currently coming from a firebase server, but i plan to change that over to use Bleve (golang) on the server. It will index all data as Services throw their data at it. Then the Ducene component can make a JSON request for the data from bleve https://github.com/blevesearch/bleve

Eventually Ducene will itself be able to index data locally in the frontend app itself. This allows mobiles that are offline to create documents and update the index and then allow the user to have them included in the search.

Then, when the mobile comes back online, the local documents that were edited / created / deleted can be pushed to the Server. On the server they will be reindexed.