Web-capable Box API; based on IndexedDB

greenrobot commented 3 years ago

To support the Flutter Web platform, we need an alternative to native libraries, as wasm does not seem to cover everything the native libs do; e.g. "proper" file APIs.

To provide a minimal valuable implementation, implement the Box API using IndexedDB (and FlatBuffers similar to how we use it in the native side). No relations, indexes etc.

Once that works, open a ticket to do simple queries next.

vaind commented 3 years ago

Yes, that shouldn't be too hard to do. I've already thought about this a little a while back and how I've seen this done in Dart is:

extract interfaces of the public APIs (Store, Box for now) instead of concrete classes
have an implementation of those interfaces depending on the platform using conditional imports

vaind commented 3 years ago

Also a thing to consider: not all browsers support IndexedDB, so if we wanted to have a complete solution, we'll need (to do) something like https://pub.dev/packages/idb_shim

greenrobot commented 3 years ago

With like 97% of browsers supporting it we might be fine.

Another thing to consider: if we'd have the lower-level implemented in Type-/JavaScript, we could use that for plain web apps too. That probably makes sense when we start adding more features like queries.

vaind commented 3 years ago

Since this would require API changes (switch to interfaces) we should try to prepare those prior to 1.0, with the actual implementation coming later as a minor release.

vaind commented 3 years ago

if we'd have the lower-level implemented in Type-/JavaScript, we could use that for plain web apps too.

notes:

to implement interaction with JavaScript (i.e. call a objectbox-js impelmentation from Dart) there's dart:js and package:js, looks like the latter supersedes the former. Regarding the performance of such an interop - couldn't find anything useful yet. From my understanding, there shouldn't be much cost to calling javascript, considering the rest of the dart code is compiled to javascript as well, maybe just some checks/data validations before crossing the boundary to the code generated by dart2js.
there's no support to integrate dart with TypeScript (maybe except if you compiled your TS to JS but I don't know how that works), see this ancient Dart SDK issue.
code written in Dart can be compiled to JS (dart2js), albeit at some size costs. From what I've read, Dart code compiled to JS should be faster than the "same" code in JS due to the optimizations Dart is able to do, e.g. because of its type system.

greenrobot commented 3 years ago

1) interfaces, yes. separate issue?

2) According to https://stackoverflow.com/a/26428441/551269, you may be able to call TypeScript just like JS. Would suggest a prototype.

vaind commented 3 years ago

you may be able to call TypeScript just like JS

As long as you write TypeScript just like JS, e.g. no generics (Box)

vaind commented 3 years ago

Turns out we don't actually need interfaces, just two variants of the concrete classes - the conditional import ensures the right implementation is imported. See #189

greenrobot commented 3 years ago

While we might not need interfaces (at least for now), it might be that we want them for the flexibility. Just tossing in that thought; more a discussion for later, I guess.

Buggaboo commented 3 years ago

I think that conditional is a compile-time thingy, it's not polymorphic dynamic binding.

Coinners commented 3 years ago

Any news?

greenrobot commented 3 years ago

While we'll keep the first steps quite simple, we should also discuss the target architecture we'll want in the end. To start this, I could imagine the following layers from top to bottom (let me know you see are technical show stoppers):

Very thin Dart wrapper API (e.g. Box)
TypeScript API (exposing a Box API, doing the interaction with the "native" WebAssembly layer)
ObjectBox WebAssembly (the core native parts and ObjectBox Sync)
JS/TypeScript HAL to provide the features we cannot use in WebAssembly
- Key/Value storage based on IndexDB
- Networking if required (not sure what wasm offers here)

While this is a more "heavy" approach, it would allow us to use the entire existing ObjectBox functionality (including Sync).

zs-dima commented 3 years ago

SQLite solution looks more reliable than IndexDB. Moor support Web with SQLite already.

vaind commented 3 years ago

SQLite solution looks more reliable than IndexDB. Moor support Web with SQLite already.

Do you have a reference for that? I don't know moor deeply but from a quick look at its web-only code it looks like it supports local storage and indexeddb as a backend: https://github.com/simolus3/moor/blob/develop/moor/lib/src/web/storage.dart

greenrobot commented 3 years ago

Regarding moor: following their docs I landed here: https://github.com/sql-js/sql.js/ which "doesn't persist the changes made to the database", which would not exactly make a good K/V storage for us…

zs-dima commented 3 years ago

@greenrobot

SQLite is ACID and IndexedDB is NOT ACID
https://github.com/sql-js/sql.js/ text follows: "allows you to import any existing sqlite file, and to export the created database" So SQLite persistence depends on implementation

vaind commented 3 years ago

SQLite is ACID

Correct, but sql-js isn't, being in-memory it's not durable.

zs-dima commented 3 years ago

@vaind https://github.com/sql-js/sql.js/ text follows: "allows you to import any existing sqlite file, and to export the created database" So SQLite DB could be saved to disk as well

greenrobot commented 3 years ago

@zs-dima I'm sorry, that does not help. I think you lack some background on the proposed stack and the purpose of the lowest (HAL/storage) layer. sql.js does not look like a fit becuse it offers SQL without storage, which is the exact opposite of what we are looking for. We rather need storage without SQL. :slightly_smiling_face:

greenrobot commented 3 years ago

We rather need storage without SQL.

Or, no storage and NoSQL :upside_down_face: Seriously, the idea of being in-memory only might become interesting, at least as an intermediate solution: we could use a native in-memory K/V. But not sure if that is really less work.

vaind commented 2 years ago

@richard457 commented in #190: Found this This if it can be of help. !

(Responding here where the architecture discussion takes place.)

That looks like a good approach if we could get the ObjectBox core compiled with Emscripten. In that case, no JS/TS wrapper would be necessary at all. @greenrobot do you think it makes sense to try this approach (compile the core with Emscripten)?

richard457 commented 2 years ago

@greenrobot did you check on the https://github.com/objectbox/objectbox-dart/pull/190#issuecomment-997431487 and https://github.com/objectbox/objectbox-dart/issues/185#issuecomment-1003434990 ? what do you think?

greenrobot commented 2 years ago

My understanding a few months back was that ObjectBox cannot run in WebAssembly unmodified due to missing mmap functionality in wasm. If people are willing to investigate more, I'd suggest to create another issue that's just about wasm.

gustavorozolin commented 2 years ago

I need migrate my App's local db and look like ObjectBox fit perfectly but my app require web support :(

Any previews for this feature?

richard457 commented 2 years ago

you can try this https://github.com/isar/isar and help me convince the author to implement sync. gustavorozolin

Macacoazul01 commented 2 years ago

@greenrobot https://github.com/isar/isar/issues/380

Innocent-Akim commented 2 years ago

lib/checkout/stripe_cheout.dart:24:12: Error: Only JS interop members may be 'external'. Try removing the 'external' keyword or adding a JS interop annotation.
external Stripe(String key);
^
lib/checkout/stripe_cheout.dart:25:12: Error: Only JS interop members may be 'external'. Try removing the 'external' keyword or adding a JS interop annotation.
external rederictToCheckout(CheckoutOptions options);
https://pub.dev/packages/js

gaetan1903 commented 1 year ago

Oh no 😅 just want run my application in web when it s x86 platform... But i use objectbox and it blocks,

So i have to remove objectbox before 😅

ThaDaVos commented 1 year ago

Curious as to what's the state of this? Or should one create an interface/adapter themselves and use something else for storage in the web?

greenrobot commented 1 year ago

A quick update on this: we are currently working on an alternative lower layer for the database. While this is not directly related solution this issue, it would fit the use case and together with wasm, we might make it work for Dart/Flutter too. This won't use IndexedDB but plain files. Don't expect anything soon yet, however...

Macacoazul01 commented 1 year ago

Curious as to what's the state of this? Or should one create an interface/adapter themselves and use something else for storage in the web?

After isar v4, probably

Paroca72 commented 1 year ago

No have a way to use the objectbox library as a memory DB? On web I understand the problem to save the data persistently but maybe make it work in memory is something feasible. In this case I could just continue to use the Objectbox library structure and sync the data with a remote server.

If I recall good Isar v4 already do something like that. Cannot store data permanently on Web but you can still use it as in-memory db.

objectbox / objectbox-dart

Web-capable Box API; based on IndexedDB #185