Implement "all" query for `EconomicResource`s

[pospi]

I'm just logging this for exploration; it will depend on whether it's needed for GFD as to whether we actually go ahead.

@sqykly has said a few times that "there is no 'all' in Holochain". Whilst this is true, there are patterns for maintaining references to collections of objects, via an "index" record which keeps an array or mapping of hashes. The caveat I have seen mentioned about this is that one must take care to avoid DHT "hotspots" by splitting up these index records to avoid particular containers being updated too frequently and thus stressing the network.

I think the MVP of such a feature would be to have an economicResources record linked from the main app DNA, so that we can always find it via {app-dna-hash}.economicResources. This record should then be synchronised with all CRUD operations to EconomicResource records so that added / changing / removed hashes are propagated to the index.

The client-side logic for loading these would be quite straightforward and basically an identical pattern to the loading of records from fixtures that has already been implemented.

[sqykly]

EconomicResources already have queries that make it a little too easy (for a production version) to collect them en masse.

• resources.getResourcesInClass: if you know a ResourceClassification hash, or even the exact name, you can get all instances of it. There are bound to be only a limited number of classifications you're interested in, so getResourcesInClass them all and you have your all.
• agents.getOwnedResources: if you know an Agent hash, you can get all the resources owned by that agent.

Anchoring with the App and Agent hashes was my first try on this issue, too - in fact, that's exactly what I did for the old chatter module. But Art warned me off of that approach citing the network weighting it imposed. I also learned that there are security implications of having such an index out in the open.

Consider that in a production app, there are likely to be multiple users. To what degree do they want to see irrelevant data from other users? To what degree do they want their own data to be available to everyone? That's the real reason that anchors are not the right tool here. The solution to partitioning user data will govern the solution to query all, and we have a lot of options there to evaluate.

I almost think our first steps in Rusty-chain ought to be (1) reproduce LinkRepo (2) come up with a similarly general purpose solution for creating "scope"s to limit access.

[fosterlynn]

(2) come up with a similarly general purpose solution for creating "scope"s to limit access.

I think part of this discussion will be how we implement the DHT architecture. If there is one DHT for each group agent, then the "leakage" between agents/users won't really be a problem (if I understand correctly). We should be able to assume a set of permissions implied by being a member of a group - so openness within a DHT. (I think. If not, we'll find that out from HC users.)

[pospi]

That sounds correct to me @fosterlynn.

Also, before we go rebuilding LinkRepo we should see if there's some prior art out there we can piggyback off. Someone else must have done it by now.

I also found this radix tree implementation, which is great because I was thinking of writing one so that we can do proper search filtering: https://github.com/skjena/trie-hadt

[sqykly]

Honestly I'll end up reproducing LinkRepo as an exercise anyway. If there's something more prepared out there, great! But I need to get my hands dirty when I learn a language, Hello World doesn't really cut it, and it's helpful to separate solving the logical problems (which I've done and know by heart for LinkRepo) from implementing the solution in a new language. I almost always re-write one of my (smaller) libraries, at least partially, as an introduction to the new language.

That doesn't mean we have to use it.

[bhaugen]

before we go rebuilding LinkRepo we should see if there's some prior art out there we can piggyback off. Someone else must have done it by now.

We haven't seen it. Art thought some other people had done something similar but it was not the same thing at all.

[bhaugen]

REA is a highly relational model so to navigate it really helps to have something that handles foreign key connections like foreign key connections.