Complete Implementation of get_related_holons dance

[evomimic]

Summary

This enhancement completes the work begun under issue #84 and delivers the "_get_related_holons_" dance to the Uniform API. This dance is a QueryMethod that retrieves references to existing or staged holons that are related to the source holon via a specified relationship_name (if such a name is supplied) or via ANY relationship if a relationship_name is not supplied. If the source holon is an existing (vs. a staged) holon, then a side-effect of this dance may be to retrieve SmartLinks from the persistent store in order to update the RelationshipTarget(s) within the source holon's RelationshipMap.

Current State of Native Functionality:

• The get_related_holons method does not currently exist.
• Some SmartLinks are being created.
  • SmartLinks are being created upon commit of a StagedHolon, but only in a limited way. Specifically,
  • Outbound SmartLinks are created for each relationship, but their inverse SmartLink is NOT being created. (We need RelationshipDescriptor support to know what the inverse relationship is). This limits navigation to a single direction.
  • Constraint-Based SmartLinks (e.g., for keys and access_paths) are NOT being created (due to the lack of support for Descriptors and HolonSpaces).
• But none are currently being retrieved.
  • RelationshipTarget has a set of cursors intended to be populated from SmartLinks, but they are not.
• The LocalCache within the CacheManager currently maps HolonId to Rc. However, _get_relatedholons needs to update the collections within the RelationshipTargets of the source holon. This means the cached holon needs to be mutable.
• We have SmartCollection and StagedCollection structs defined, but without much associated behavior
• We DON'T have HolonCollection defined.
• SmartCollection and StagedCollection both have keyed indexes, but they are not being updated

Dependencies

• [x] Issue #89
• [x] Issue #84
• [x] Issue #105
• [x] Issue #106
• [x] Issue #114
• [ ] Issue #88

Proposal

Enhance Native Functionality

Define and Implement get_related_holons method to the HolonGettable Trait

fn get_related_holons(
        &self,
        context: &HolonsContext,
        relationship_name: Option<RelationshipName>,
    ) -> Result<RelationshipMap, HolonError>;

In this method,

• &self is a either a HolonReference, StagedReference, SmartReference or Holon that represent the source holon whose related holons are being requested
• relationship_name, if provided, indicates the name of the relationship being navigated. In the future, this parameter will be replaced with an optional reference to the RelationshipDescriptor for this relationship. If None, then all holons related to the source holon across all of its relationships are retrieved.

This method populates the cached source holon's RelationshipTarget for the specified relationship if one is provided. If relationship_name is None, the source holon's RelationshipTargets are populated for all relationships that have related holons.

In the holon.rs file

• [x] Add the following method to the HolonGettable Trait:
• [x] Implement the get_related_holons method for Holon

_In the holonreference.rs file

• [x] Implement the get_related_holons method of the HolonGettable Trait. This implementation should just delegate the call to the same method on its variant.

_In the stagedreference.rs file

• [x] Implement the get_related_holons method of the HolonGettable Trait.

_In the smartreference.rs file

• [x] Implement the get_related_holons method of the HolonGettable Trait.

Dance Enhancements

Implementation Initial Query Layer Data Objects

This issue provides the opportunity to introduce the Query Layer of the guest-side architecture:

The basic concept is a graph data structure consisting of nodes, relationships, and collections . Nodes, their relationships, and the collections referenced from those relationships are returned as results of query dances and those nodes or collections can be used as inputs to subsequent query dances.

• [x] Define Initial Query Layer Data Structures:

  
  pub struct Node {
  source_holon: HolonReference,
  relationships: Option<QueryPathMap>,
  }

pub struct Collection { members: Vec, query_spec: Option }

pub struct QueryPathMap (pub BTreeMap<RelationshipName, Collection>)

pub struct QueryExpression { relationship_name: Option }

Note that none of these objects actually contain Holon's themselves, only references to holons. Holon state is only maintained in the _Shared Objects Layer_. Specifically, each staged Holon exists exactly once in the _StagingArea_ and each previously persisted holon exists at most once in the _HolonsCache_.

**_In the dance_request.rs file:_**

- [x] Change the DanceType::QueryMethod variant definition to: `QueryMethod(Collection),`
- [x] Add a `RequestBody` variant of `QueryExpression(QueryExpression)`

**_In the dance_response.rs file:_**

- [x] Add a `ResponseBody` variant of `Collection(Collection)`

_**In the holon_dance_adapter.rs file:**_

/// DanceRequest: /// - dance_name: "query_relationships" /// - dance_type: QueryMethod(Collection) -- specifies the Collection to use as the source of the query /// - request_body: QueryExpression(QueryExpression) /// /// /// ResponseBody: /// - Collection -- a collection containing the same source nodes passed in the request, but with their relationships /// updated to include entries that satisfy the criteria set by the QueryExpression supplied in the request ///

- [x] Add a new `build_query_relationships_dance_request` function
- [x] Implement a new `query_relationships_dance` dance function  
This function iterates through the supplied Collection and invokes the `get_related_holons` method on the node's `holon_reference` and transforms the resulting RelationshipMap as follows:
- create a Node whose `source_holon` matches the HolonReference if the input node and whose relationships are populated from the query result as follows:
    - For each entry in the RelationshipMap returned by the native function, add an entry in the Node's `relationships` map. The `RelationshipName` key can be cloned directly from the RelationshipMap. To create the `Collection`, iterate through the _HolonCollection_, create a `Node` for each `HolonReference` and add that node to `members`.

_**In the dancer.rs file:**_

- [x] Add "query_relationships" dance to the Dancer's dispatch table

## Testing Enhancements

_**implement a new test_query_relationships.rs file:**_

- [x] Implement "execute_query_relationships" test step executor

_**In the test_data_types.rs file:**_

- [x] Add a QueryRelationships(Collection, QueryExpression,  ResponseStatusCode) variant in the `DanceTestStep` enum definition
- [x] add its fmt::Display implementation
- [x] implement the `add_query_relationships_test_step` function that takes a Collection, a QueryExpression, and expected ResponseStatusCode

_**In the dance_fixture.rs file:**_
- [x] Edit the existing `simple_add_related_holons_fixture` 
- [x] Add a `query_relationships` step to the test_case after the commit step. Expect OK and check that the previously related holons are returned

_**In the dance_tests.rs file:**_
- [x] Add a match for the  `DanceTestStep::QueryRelationships` step that passes `node_collection`, `query_expression`and `expected_response` to the `execute_query_relationships` function

### Carryover from #79 

- [x] Enhance the _abandon_staged_changes_ test case to use get_related_holons to ensure:
    - [x] relationships that are expected to have been populated on commit ARE populated and relationships that are NOT expected to have been populated have NOT been populated.
    - [x] attempts to get_related_holons for an abandoned holon fail

## Definition of Done:
- [x] After adding `query_relationships_test_step` to the `simple_add_related_holons_fixture`, test case **passes** 
- [x] After adding  `query_relationships_test_step` to the `abandon_staged_changes` test case, test case **passes**

[dauphin3]

@evomimic if there are not relationships for the given name, should get_related_holons just return an empty RelationshipMap, or should it be an Option or HolonError ?

[evomimic]

@evomimic if there are not relationships for the given name, should get_related_holons just return an empty RelationshipMap, or should it be an Option or HolonError ?

Empty RelationshipMap

[evomimic]

Comments on this morning's push:

• HolonReference implementations of get_related_holons looks good
• StagedReference implementations of get_related_holons look good, although a few comments in the code about the strategy behind that implementation might be nice.
• SmartReference implementation is incorrect. In fact, I just realized that most of the functions implemented in SmartReference are doing work that should be delegated to their referenced holon.

Proposal

_In smartreference.rs:

• [x] Factor out the repeated code for getting an _rcholon from the context into a private get_rc_holon(&self, context)->Result<Rc<RefCell<Holon>>, HolonError> method
• [x] Delete the get_key_from_link method (it doesn't seem to be used -- and this is not the right place to be mucking around with link internals anyway).
• [x] get_property_value() -- if it can't get the property value out of smart_values, it should use get_rc_holon to get the rc_holon and then delegate the get_property_value call to the rc_holon.
• [x] 'get_related_holons-- useget_rc_holonto get the rc_holon and then delegate theget_related_holons` call to the rc_holon.
• [x] get_key -- should first attempt to get the key from its smart_property_values and, failing that, use get_rc_holon to get the rc_holon and then delegate the get_key call to the rc_holon.

The get_property_map and get_relationship_map methods should just be deleted, but this has ripple effects to other files, so I've pushed those changes off to a separate issue #118.

There are also several issues in the holon.rs file: <I'll fill these in shortly>

[evomimic]

Some additional comments on the previous push:

• Holon implementation ofget_related_holons is incorrect. It should be calling load_relationship_map.
• holon.rs defines, but does not actually implement the HolonGettable Trait. It has implementations for its functions, but they are in the impl Holon block instead of a separate impl HolonGettable for Holon block.

My review also revealed a problem with the load_relationship_map implementation. One of the central tenets of the Shared Objects Layer is at-most-once semantics. Specifically:

1. a holon_node should be retrieved at-most-once from the persistence tier and then cached and
2. the links for a holon_node should be retrieved at-most-once from the persistence tier.

In the current implementation, (1) is enforced by HolonCache, but (2) should be enforced by load_relationship_map, but it isn't.

Proposal

_In smartreference.rs:

The get_property_map and get_relationship_map methods should just be deleted, but this has ripple effects to other files, so I've pushed those changes off to a separate issue #118.

There are also several issues in the holon.rs file:

• [x] delete the get_holon(id) method (and delete the call to it from holon_api.rs).
• [x] Move the functions that implement HolonGettable into a impl block for HolonGettable. Note: this may require adding _context as a parameter to match the Trait signature.
• [x] Move the get_relationship_links function to smartlink.rs and change its signature and implementation so that it returns a Result<Vec<SmartLink>, HolonError> -- As a general rule, there should be no visibility or use of holochain LinkTypes outside of the smartlinks.rs file.
• [x] Enhance the load_relationship_map function to only invoke get_relationship_links once per relationship
  • make load_relationship_map a method on Holon that takes &mut self instead of holon_id
• Before calling get_relationship_links for a relationship_name, we should first check the relationship_map to see if it already has an entry for that relationship. If so, simply return that entry.
  • If not, call get_relationship_links and add an entry in the RelationshipMap for that relationship_name -- even if no links for found for that relationship. This keeps us from calling get_relationship_links again for that relationship (honoring at-most-once semantics).
  • NOTE: This still leaves one outstanding issue. There is no way to tell if we have retrieved ALL of the relationships for a Holon without adding some sort of state field to RelationshipMap.

[evomimic]

Steps for completing the carryover from Issue #79 . These steps are being deferred to a later issue.

• Enhance the abandon_staged_changes test case to use get_related_holons to ensure: relationships that are expected to have been populated on commit ARE populated and relationships that are NOT expected to have been populated have NOT been populated.

In the holon.rs file:

• Extend the definition of EssentialContent to include the RelationshipMap

In the dance_fixtures.rs file for the abandon_staged_changes test chase:

• After the commit step, we need to add a query_relationship_test_step. This step requires a NodeCollection containing the source nodes for the query and a QueryExpression containing the relationship_ name of the relationships we want to query.

  • NodeCollection: should contain a Node containing a HolonReference to the Book holon we just committed
  • QueryExpression: should specify the "AUTHORS" relationship_name.

• Add a match_db_content_step that matches the Book holon to the retrieved Book holon

• attempts to get_related_holons for an abandoned holon fail