Basic Query (Builder) support

[greenrobot]

Goal: minimal solution including a query builder and a reusable query object. For reference, please check the Java query API.

• [ ] QueryBuilder to allow "equal" criteria for string and int properties (pass property IDs to identify properties if we don't generate a meta model yet)
• [ ] QueryBuilder allows building a Query object
• [ ] Query object methods: find() and findFirst(). find() returns all matching objects, findFirst() just the first one.

[Buggaboo]

I started by wrapping the C functions, I'll share a branch soon.

With regard to designing the interface, I'm thinking of passing a closure to the Store#query method, to produce a QueryBuilder object like swift.

I think another way to do it idiomatically in dart is by declaring this query method as a factory that takes a closure that takes instances of Condition in the body. Also one can leverage the .. cascade operator, so there's no need to declare those types as part of a builder pattern. And boolean operator overloading for any / join (i.e. respectively ||, &&).

At some point in the future, we'll can leverage extension (a la Objc: category, swift: extension), to bolt on the properties to create conditions. Right now, I'm generating the EntityType properties, in a separate EntityType_ class (like java).

@Entity class Person { @Id... }

final store = Store([...]);
final box = Box<Person>(store);
final query = box.query(() {
    Person_.age.isNotNull()
        ..greaterThan(18); // equivalent to: && Person_.age > 18
}).build();
var p = query.findFirst();

Something like this.

[vaind]

Factory method taking Condition is what Go does (not closure, directly a list of conditions). That's probably similar to what you had in mind, isn't it? Maybe it would make the code/usage simpler without the closure?

box.Query(Person_.age.greaterThan(18), Person_.name.startsWith("John"))

[vaind]

A few things to keep in mind:

• consider building queries an expensive operation, the idea is they should be built once and reusable (just changing parameter values)
• there are links (over relation fields) - swift and go take slightly different approaches, with Go creating a condition and passing it to the query and swift chaining the link on the query object (therefore requiring an explicit build() call which Go doesn't need)

In any case, having a look at the query-builder implementation in go/swift may help you clear some things up. https://github.com/objectbox/objectbox-go/blob/master/objectbox/querybuilder.go

[vaind]

• java - https://docs.objectbox.io/queries
• go - https://golang.objectbox.io/queries
• swift - https://swift.objectbox.io/queries

[Buggaboo]

We can't use varargs or reuse method names, that's why I initially thought of passing a closure. For example:

QueryBuilder Query(Condition... c)

Nor this will work:

QueryBuilder Query(Condition c1) { ... } 
QueryBuilder Query(Condition c1, ..., Condition cn) { ... }

We can do optional positional arguments though of constant size n, but I doubt that's gonna be pretty.

[vaind]

Oh, no variadic functions in dart? What about taking a list then? Still less cumbersome than a closure

[vaind]

And there's a "solution" for variadic functions as well actually: http://yuasakusa.github.io/dart/2014/01/23/dart-variadic.html

It would be interesting to see the performance hit when compared to passing a list

[vaind]

Actually, looking back at your original comment @Buggaboo, I quite like the idea of operator overloading. However, is it really necessary for Query to take a closure? What about taking a Condition interface, i.e. something produced by Person_.age.greaterThan(18) or by an operator overload (a combination of two conditions) - Person_.age.greaterThan(18) && Person_.name.startsWith("A")

[Buggaboo]

I have something working, it might leak or explode on your machine. I'm not ready to merge yet, but count already works.

This design allows for very complex Condition trees using and / or. It would be nice if the user wouldn't need to manually close a query.

Also I'm not very sure of the type mapping I have right now:

{
    "dart" : "objectbox-c"
    "int" : "Int64", // this could be Int32 instead, or even a c-byte (aka Uint8), dart<2.0 used to be arbitrary precision
    "bool" : "Uint8",
}

I added a box.queryAll and box.queryAny function, e.g.:

final text = TestEntity_.text;
final anyGroup = <QueryCondition>[text.equal("meh"), text.equal("bleh")];
final queryAny = box.queryAny(anyGroup).build();
// equivalent to
box.query(text == "meh" | text == "bleh").build();
// equivalent to
box.query(text.equal("meh").or(text.equal("bleh"))).build();

[vaind]

I've taken first a look at the code and it looks good :+1: I especially like the operator overloads.

There are a few things that could be updated with the changes introduced by https://github.com/objectbox/objectbox-dart/pull/25 - ~I hope it gets merged soon.~ It's merged into dev now.

Also, as you mentioned, it seems like #11 is a prerequisite of queries supporting all types. However, it doesn't mean we can't finish the rest first and maybe even merge it in.

Do you mind creating a PR? It's fine if the code is unfinished but at least some parts could be clarified already.

[Buggaboo]

I'll do a PR after I implement findFirst / find / findIds. I quickly implemented findFirst and find by (ab)using Box<T>.getMany.

I plan to expose new OBXFlatbuffersManager<T> in a certain Box<T> to the queries, to reuse for unmarshalling the buffer from find. Instead of passing the whole box like I do right now.

I haven't started on orderBy yet.

[vaind]

I plan to expose new OBXFlatbuffersManager<T> in a certain Box<T> to the queries, to reuse for unmarshalling the buffer from find. Instead of passing the whole box like I do right now.

Yes, makes sense for queries to have access to their entity's box. Similar is done in Go.

I haven't started on orderBy yet.

Order, offset, limit, params, etc. are not necessary for the first step. Let's limit the scope of this issue to "Basic query support" so the PR would be more manageable if you don't mind.

[GregorySech]

Hi, I'm not sure if this is still a matter of discussion however I wanted to share a couple of Dart projects with built-in ORMs. I guess that a major part of the objective here is having an API similar to the other languages SDKs but maybe some inspiration can be drawn from these:

• https://moor.simonbinder.eu/docs/getting-started/writing_queries/
• https://aqueduct.io/docs/db/executing_queries/ not Flutter compatible, it uses reflection.

Edit: Maybe the aqueduct one is a little out of scope as it isn't compatible with Flutter.

[Buggaboo]

Thanks, the stream and futures look nice; there are async stuff on the objectbox-c project, that can be implemented to support async queries. The syntax is somewhat constrained by how objects/methods are defined in the objectbox-c project.

There should be a new ticket for syntax-sugar related wish-list. Since objectbox is not a SQL DB, I don't feel that we have to stick to that syntactic tradition, e.g. select(cond)..groupBy(prop)..orderBy(prop) etc..

I can imagine something like this:

// box = Box<S> bla
final builder = box.query(S_.text == 'meh');
final firstFuture = builder.firstFuture(); // Future<S>
final listFuture = builder.future(); // Future<List<S>>
final generator = builder.generator(); // Iterator<S> Function();
final stream = builder.stream(); // Stream<List<S>>
// Stream<S> singleStream = builder.streamSingle(); // is probably achievable thru a StreamTransform.
try {
  await print(firstFuture.toString());
  await for(list in listFuture) {
    print (list.map((s) => s.toString()).toList().join(", "));
  }
  print (generator()); // yields s0
  print (generator()); // yields s1 after s0
  print (generator()); // yields s2 after s1
  await print(stream.first);
  stream.listen((list) => print (list.map((s) => s.toString()).toList().join(", ")));
} catch (err) {
  print('Caught error: $err');
}

We don't really have groupBy support yet. Although this can be implemented from dart.

[GregorySech]

Would be really nice, about aggregators I guess it's fine. For what is worth I really dig the idea of using operator overloading to logically join the Conditions.

[vaind]

@GregorySech

I'm not sure if this is still a matter of discussion however I wanted to share a couple of Dart projects with built-in ORMs. I guess that a major part of the objective here is having an API similar to the other languages SDKs but maybe some inspiration can be drawn from these:

Thanks, it's pretty similar to what we do in other languages as well but when there's a specific functionality that needs to be done differently, it's always nice to have a fairly "standardized" interface - that's where it makes much sense to look at other solutions - if it makes sense to provide familiar experience to developers.

[vaind]

@Buggaboo

We don't really have groupBy support yet. Although this can be implemented from dart.

FYI, aggregators are (to some extent) supported by objectbox-c = Property-Query OBX_query_prop - that would end up as a new issue I guess, certainly not in scope of this one.

Since objectbox is not a SQL DB, I don't feel that we have to stick to that syntactic tradition, e.g. select(cond)..groupBy(prop)..orderBy(prop) etc..

Certainly - while there are similarities, ObjectBox doesn't even want to look/work like SQL.

[vaind]

Thank you @Buggaboo for tackling this. It ended up to be quite a big PR :+1:

[Buggaboo]

Well, we did it together more or less. Now we have proper ffi Structs etc., and other fancy stuff. I learned a lot.

[greenrobot]

Awesome! :+1: