Closed sharmaneil closed 3 years ago
For data storage / sync / transfer I recommend we use existing solutions rather than rolling our own. Implementation would require changing the way we store documents a tiny bit, but would be overall much easier. Examples include:
If possible it probably makes sense to just default to any such system that our devs have experience with, if anyone else has used one of these?
Thanks for the initial research.
I haven't used any of these, but we can check with the the new engineers next week. Of the options listed, Pouch and RxDB seems interesting! It's a bit hard to evaluate without playing around with them.
I've used firebase way back when for real-time syncing, but sounds like google may be out of the question. I've used dynamodb for tiny scrappy projects, but not for any real-time work.
If it helps to evaluate, here are some of the queries we'd need:
MVP must-haves:
MVP maybes:
Post-MVP:
Theoretically on the must-haves, those can all be sync based operations (sync from local). I think most of the maybes should be the same (also doable via syncing local data). That makes this a lot easier.
AWS also has appsync https://aws.amazon.com/appsync/?sc_ichannel=ha&sc_icampaign=acq_awsblogsb&sc_icontent=mobiledev-resources
Might be a little more work to setup for an MVP tho, but could be a better long-term solution
Some of this is already in other tickets so I'm going to pull them out of this for now:
User Management / Auth is here: https://github.com/CDCgov/prime-central/issues/89
Analytics (placeholder) here: https://github.com/CDCgov/prime-central/issues/67
@sharmaneil not sure we can guarantee our hosting platform will be AWS. @RickHawesUSDS has more details here. But wanted to warn you early on.
In that case, I'm going to try out one of the databases in Pete's suggestions instead of picking something AWS-specific. If we don't decide soon, it'll create tech debt if we ever want to switch databases, but that may be unavoidable.
For our non-prod apps, cloud.gov seems to have a limited set of managed database options:
Don't know what the LoE is to spin up a different database
@sharmaneil Suggest Postgres because it is open-source, works on dev machines, and can be deployed anywhere including the environments being considered. You can also just store JSON in Postgres, if that makes things go faster.
Building off of the recs and research so far, here's an idea:
hasura
- an open source Backend as a Service to create graphql APIs
RxDB
- this is a wrapper around client-side databases. It uses observables, which is a javascript programming paradigm that deals with streaming data. Not wed to anything; we can discuss more early next week.
Another option for offline support: Redux Offline
Pros:
Cons:
Some of these options still look like building a lot of stuff to make them work (i.e. Redux Offline seems to primarily be built to handle network resilient API calls rather than data synchronization out of the box, so we have to build that API). I think we'd be best off avoiding anything more complex that basic data sync for early stages, but we keep coming back in this direction so maybe that's the way we need to go.
@pete-usds - can you give an order of magnitude or time difference for what it means to build a lot of stuff vs. not build a lot of stuff? Want to make sure we're coming from the same set of assumptions about what is acceptable vs. not.
I think we should have a synchronous discussion with the expanded eng team today/tomorrow to see what skillsets folks have and figure out what path makes sense. Doesn't seem like we're moving toward a consensus, and several of the engineers who'd be working on this aren't in this thread.
I'd like to roll out a locally-running solution to all of this -- wired to the frontend -- in this sprint, so making a decision soon here would be helpful
@sharmaneil I think GraphQL makes a lot of sense for the PDI backend. No experience with Hurusa, but it looks like a fit on the surface.
Did the synchronous discussion happen? Also, let me know if I can be helpful on:
- [ ] shared understanding of what the priorities are for the next prototype version (ie: multi-user, sync, offline, etc)
Overall my number 1 priority is avoiding data loss of test results. That would completely destroy trust in our app.
Based on the latest discussion with @RickHawesUSDS @benwarfield-usds and @nickrobison-usds, the tech stack decision is:
Final acceptance criteria checkbox is addressed by this Epic from Alicia: https://github.com/CDCgov/prime-central/issues/105
tl;dr: multi-user sync, but not offline.
Going to go ahead and close this out, woo!
The prototype right now is a static-site with no backend. Depending on what the MVP looks like, we'd need a solution for:
Acceptance criteria:
Updated Nov 2, 2020
Questions to answer:
In other tickets: ~- persisting users~ ~- authenticating users~ ~- logging/analytics~