[Tracker] Collections library

[gkc]

Is your feature request related to a problem? Please describe.

Currently every app developer has to independently implement the glue code required to join their app's domain objects (and lists of those objects) with put/delete calls to the AtClient API on the one hand, and update/delete notifications from the notification service on the other.

Buzz and the insurance poc app are current examples of this. We have an opportunity now to take the learnings from those apps and implement a set of classes which could be used by any app and which will cover the typical 90% which is usually CRUD (Create, Retrieve, Update & Delete) but which for the atPlatform we will cover as SCRUD (Share, Create, Retrieve, Update & Delete)

This leads to a lot of extra effort on the part of developers, with some concepts being non-obvious (e.g. "sharing" with an atSign means calling AtClient.put(sharedAtKey, value) which results in the creation of an entry in the keystore, with the data encrypted specifically for that atSign, and "un-sharing" means calling AtClient.delete(sharedAtKey) where sharedAtKey is something like @bob:12345.addresses.myapp@alice)

Describe the solution you'd like

Set of (generic) classes which provide glue code which could be used by both Buzz and the insurance poc app

Some initial ideas to seed the design conversation

• Something like a Collection class, which collaborates with an AtClient and NotificationService in some way - possibly directly, preferably through an abstraction like AtDataService
• AtDomainObject abstract class should have the minimum information required to interact with the platform. This may be just a unique ID, but may well also include an AtKey
• The Collection should know what part of the atSign's namespace it maps on to. For example a Collection of Addresses might live in addresses.myapp@atSign with individual objects having atKeys like
  • 12345.addresses.myapp@alice (self key)
  • @bob:12345.addresses.myapp@alice (the copy shared with bob)
• The Collection should provide
  • Retrieve - retrieve a list of the items in the collection with ability to filter, and ability to make the data 'live' by setting up the required Notification KeyStream(s)
  • Create, Share, Update, Delete, UnShare
• We need to be able to handle AtDomainObjects which contain other AtDomainObjects (e.g. a Company containing an Address) in a smart way, by reference rather than by value. This matters when the AtDataService marshalls before doing put()'s and what it unmarshalls notifications it receives, and in several other places.
• Ideally a single logical thing like an Address would have a single copy in the app's memory

### High-level tasks (please add additional tasks here!)
- [ ] [Clearly define what we want to achieve via design doc and/or code prototype](https://github.com/atsign-foundation/at_client_sdk/issues/797). (Take what's been learned from Buzz and the insurance POC, merge it with some of the ideas / objectives outlined above, write a detailed design doc and/or code prototype and test cases illustrating the things that we want to be true, get it reviewed)
- [x] Defining abstract classes and pseudocode (https://github.com/atsign-foundation/at_client_sdk/issues/826)
- [ ] High-level marketing / launch plan (how do we talk about this, why it's important, why it's cool, how to use it, etc)
  - [ ] Timing Mar 2023
- [ ] Initial implementation
- [ ] Refactor insurance POC to use these new "Collections" abstractions
- [ ] Modify Collections implementations as required based on what didn't fit / wasn't right for the insurance POC
- [ ] Refactor Buzz to use the Collections abstractions
- [ ] Modify Collections implementations as required based on what didn't fit / wasn't right for Buzz
- [ ] "How to" guides on docs.atsign.com
- [ ] Text or video (or both) content for posting on social or our website

[XavierChanth]

Thoughts from an earlier discussion: https://gist.github.com/XavierChanth/f7f92701657db773b5d42158c014d8d4

Meeting minutes 17/11/2022:

• Main goal, to avoid writing boilerplate code
• What shape does this take?
  • If you load the same object into memory 5 times, do you want 5 copies? or do you want 1?
    • Assumption: Same id = Same data
  • Ability to refer to other objects
  • Ability to write it in pure dart
  • Developers should not have to deal with the atPlatform to be able to use it More specifically, they should understand what's happening, but not need to worry about it.
    • If there are 5 copies of AtKeys being stored (1 for each sharedWith), it should still feel like one
• Can we remove the need for state management in Flutter entirely?
  • No, we have to reload data each time we reload a screen
• Can KeyStreams be used?
  • Need to support filtering for efficiency.
• Mental model improvements:
  • Rename AtDataService to "AtCollectionController"
  • Rename CollectionModel to "AtCollectionModel"
    • Allow "AtCollectionModel" to do call operations on itself
    • update, delete, share, etc... (not create, that will have to be on the controller or static)
• What is atomic? In terms of the app
  • Store intents, app crashes, retry on load
  • ex. intent to create 500 things
    • How do we state an intent for these things to happen?
    • When you have a single "business" operation but in reality several smaller operations:
    • Create an intent to update something, then reshare it with x # of atSigns
    • Should intents be available on any client?
      • Problem with race conditions...
      • Simpler to stick to a single client and handle intents locally
    • We could, upon restart, require confirmation to handle intents.

To be discussed:

• References to models and back references
  • How to serialize objects that contain other objects

[VJag]

Here is the detailed plan. Idea is to deliver V1 in PR52

Version 1

• [x] Define and document core abstractions of the collection framework
• [x] Implement the following functionalities to the object models in a stand-alone/non-recursive way
• [x] Save, Share, Retrieve, Delete, Unshare, Update
• [ ] Add support for intents and actions for pending intents
• [x] Demonstrate and compare the simplified usage of the above methods
• [x] Demonstrate the usage
• [x] Unit tests, PR and merge

Version 2

• [ ] Define and document handling of parent-child relationships (Focussed on buzz and other known use cases)
• [ ] Implement handling of parent-child relationships
• [ ] Demonstrate the usage
• [ ] Unit tests, PR, and merge

Version 3

• [ ] Define and document handling of one-to-one, one-to-many, many-to-one, and recursive relationships
• [ ] Implement handling of parent-child relationships
• [ ] Demonstrate the usage
• [ ] Unit tests, PR, and merge

[sachins-geekyants]

In this sprint, we have added end-to-end test, refactored code and unit tests, and added docs. Moving to the next sprint.

[sachins-geekyants]

Raised a draft collection PR : https://github.com/atsign-foundation/at_client_sdk/pull/906

If testing is good, we can proceed with merging it.

[sachins-geekyants]

Reducing the SP to 5 for testing.