DataStore: support for multi-tenant apps with sharing?

[tslocke]

As of today, the DataStore syncs the whole DataBase

(From https://github.com/aws-amplify/amplify-js/issues/4957#issuecomment-589760076)

Could I suggest this information be presented front and centre at the start of the docs? This is fundamental to understanding what the offering is here. I stumbled across the above issue by accident before I invested time learning Amplify DataStore and I'm very glad I did!

This seems like a significant departure from the architecture of a typical "data in the could" app, and I'm struggling to understand how it can be practical for many kinds of apps.

To take a simple example -- a multi-tenant "to-do list" app -- I would normally expect to have a single database on the server. There would be user-ids on the to-dos or the lists. Obviously having each web/mobile client sync everyone else's to-dos doesn't make sense.

Maybe this would be handled via auth rules? Maybe I create isolated databases for each user? I haven't found either in the docs so far.

Both of those break down in the case of sharing. Maybe I want to delegate to-dos to other users. Maybe it's a tumblr like app, where everything is readable by everyone. Clearly, I still don't want each client to sync everyone's posts.

Am I missing something?

Thanks!

[cto-leaps]

@grant-d What solution did you use une the end? Any package in particular?

[cto-leaps]

@grant-d (I'm still toying with the idea if dropping Amplify and moving to Firebase...)

[grant-d]

Unfortunately nothing easy - we literally rolled our own sync using websockets and watermarks and nuts and bolts and chewing gum (it's not the first time my team has done this, so we knew to avoid this but ...) So nothing reproducible unfortunately - I can't give you a useful package or the like. Though we did look at Watermelon db briefly, it has a customizable sync adapter pattern that looked promising. We'd already started down the custom path though so didn't get into it. May be worth investigating. https://nozbe.github.io/WatermelonDB/Advanced/Sync.html

[Edit] As an aside, we've loved everything else about Amplify (GQL, resolvers, etc) vs Firebase. This is the only 'broken' component we've dealt with so far. What we found on Firebase is that time-to-solution was incredibly fast but we hit the ceiling pretty quick once we needed to do advanced stuff (plus the risk of Google dropping it.) Amplify is more nuts-and-bolts which means a steeper learning curve but much more headroom. YMMV.

[danrivett]

Thanks @grant-d. We're in the process of a last minute redesign to get around this. Our current intention is as follows (I'll describe it in case it helps others or they see holes in it and help us!):

Rather than have one model record which has 1 owner with CRUD access and several approvers with read-only access (using Amplify's documented @auth rules) we're going to have a separate model record for each approver. A backend lambda will mirror out changes from the master record to create and sync these shadow records, one per approver using a new masterId field stored on the record to correlate back to the master record.

Creating these duplicate records allows each record to have a single owner and so data will be populated and updated by DataStore for each owner and approver. That's the idea, we'll see once we've implemented it as just started it today.

Creating the backend lambdas to mirror the data between the master record and approver records I'm sure is not going to be trivial, with edge cases and race conditions, and candidly a bit ridiculous that this usage pattern forces such hoops to be jumped through, but at the 11th hour that we're in, this seems based on our current understanding, the best option for our particular use case.

[grant-d]

We considered the same approach - massive denormalization of records - we internally called it spraying the data. Besides the pain of needing to do this in the first place, we considered that a viable design. There's a hit on storage, latency and cpu/mem (lambda) cost of course but depending on how you do it, it may be negligible. As you suggest, it aligns with eventual consistency, and you could argue the design is semantically equivalent to sql's materialized views regardless. [Edit] I remembered why we didn't do it - the cardinality in our specific case was too high - each record could be sprayed N times, where N was theoretically unbounded. The lambda sprayer was going to be lots of fun.

[sfratini]

Not Amplify related but we eventually went with a "manual" API Gateway + Lambda approach using serverless instead of Amplify. It seemed the defaults were not enough for our use case and we wanted to have more granular control.

The auth is handled by Postgres RLS partially, to handle the tenant part.

[pokey]

After abandoning amplify due to all the above reasons, I then tried realm, then firestore, then firestore+watermelondb, before eventually landing on firebase realtimedb+serializr+mobx. I'm actually really happy with the final setup. Had to write some generic glue code to make the three libs play nicely together, which I may open source if there's interest. Certainly not battle-tested yet but seems pretty solid so far and none of the headaches I've had with any of the other frameworks I tried

[de1mat]

@pokey I would be very interested in any further details on your journey and any code you can show. Thanks for sharing - would make an interesting blog post, along with many of the other useful comments on this thread.

[laurenzfg]

Huge +1, this has been a show-stopper for my app, too.

[github-actions[bot]]

This issue has been automatically locked since there hasn't been any recent activity after it was closed. Please open a new issue for related bugs.

Looking for a help forum? We recommend joining the Amplify Community Discord server *-help channels or Discussions for those types of questions.