search

googleapis / nodejs-firestore

Node.js client for Google Cloud Firestore: a NoSQL document database built for automatic scaling, high performance, and ease of application development.
https://cloud.google.com/firestore/
Apache License 2.0
642 stars 149 forks source link

Eager loading Document References #255

Open Tyranwyn opened 6 years ago

Tyranwyn commented 6 years ago

Hey,

It would be very handy to be able to eager load Document References as objects. Is this something you have considered?

Thanks anyways :P

schmidt-sebastian commented 6 years ago

Hi @Tyranwyn! Can you elaborate a little more what your use case is? If you are interested in pre-fetching of documents, then you will likely want to use one of our clients that supports offline persistence (such as our Mobile Web client). The Server SDKs do not cache any data.

That being said, you will have to pre-fetch documents manually in the Mobile SDKs as well. If you would like us to add this as a top-level feature, you can find out team at https://github.com/firebase/firebase-js-sdk

Thanks!

Tyranwyn commented 6 years ago

Oh, thanks for your quick reply. I have modelled the database so that some objects contain references to other objects, and it would be awesome to have an option to get the referenced objects themselves instead of just references. Im displaying a list of "Employees" and they reference to an array of "Project" objects. Now I have to loop over all the employees, and then loop over the array to get all project objects. It would be easier (and more efficient?) to get these projects as an object stored in the array of the employee.

I hope this is not too confusing? Also, this probably needs to be implemented client-side aswell as server-side? Tell me if I'm wrong.

Thanks!

schmidt-sebastian commented 6 years ago

Oh, interesting. That's a feature request we don't yet have on our roadmap. I can't promise that we will get to it soon, but we will discuss this internally and see if it makes sense to expose it in all of our clients.

davidtlee commented 6 years ago

I would love something like this too! Perhaps something like the Eager Loading Syntax for Rails? It would be great for minimizing N+1 queries.

A first step might be if it were possible to efficiently support Logical OR queries and to have syntax that allows you to pass in an array for matching anything in that array, e.g. where('projectId', 'IN', [1,4,10,15,20]). Then the client could do the eager loading themselves.

BenWhitehead commented 5 years ago

I played around with this a bit the other day, and I think you can do this yourself with the existing client. Here is a gist I made illustrating how I did it.

Granted it's more than one line, but it also gives you more control of the traversal and fine grained error handling/recovery options.

Given that firestore is a collection oriented document store, it isn't really in the same area as relational databases in terms of providing support for joins.

jakeleventhal commented 3 years ago

bump

schmidt-sebastian commented 3 years ago

We would need some more user feedback before scheduling this work.

davidtlee commented 5 months ago

@BenWhitehead my understand is that your code for doing it yourself would not resolve N+1 queries. for example, if you wanted to get a bunch of entities from "x", and then for each of them, query entities from "y", you would need to make a query for every single result from x (whereas in eager loading, that would be done on server side so there are not so many separate requests)

It does look like firebase has added 'in' queries though now (with some limits) that can now make it at least possible to reduce N+1 queries to N/30 + 1 queries which is probably good enough for most cases.