Understand, document and clean up memory

[kavyasrinet]

Type of Issue

Select the type of issue:

• [ ] Bug report (to report a bug)
• [ ] Feature request (to request an additional feature)
• [x] Tracker (I am just using this as a tracker)
• [ ] Refactor request
• [ ] Documentation Ask

Checklist

Use this checklist if you are using this issue as a tracker:

• [ ] How to use basic_search, more examples of keywords
• [ ] Clean up in for example mc_memory_nodes -> explain attributes of tables in a doc perhaps. Things like : NODE_TYPE,
• [ ] Consistent naming for things like : ref_type
• [ ] Mention in the doc that BlockObjects and InstSegNodes are under the same table ReferenceObjects
• [ ] Should we get rid of BlockObjects for MC?
• [ ] Right now we have basic_search as well as get_filters and get_filters_from_lf used throughout - we should stick with one consistent way of querying memory for a filters subdict for our use cases.
• [ ] Document what possible ways and APIs there are to read / write / query memory
• [ ]
• [ ]
• [ ] Task 2
• [ ] Task 3 ...

[kavyasrinet]

High level WIP doc: https://docs.google.com/document/d/1XBKaYrcYLsP2XwMFryKKGGcIgT4BA4HTq2G75ohUmmU/edit

Also refer to: https://docs.google.com/document/d/1LNbwt_FF-v6TQXRC2wCjMSrsnBnPQBCDAlvx3uy6B50/edit

Questions:

1. memory API is spread across: memory_filters.py + (sq_memory.py + agent's memory.py) right ?
2. what high_level purpose does memory_attribute serve ? From looking at it, looks like an abstraction (to perform one level complex stuff) on top of memory APIs ?
3. memory_nodes is just all node types where each node is a class with certain attributes and functions - most of which are getters, but we also have create, update() and some attributes here are exposed. For example, name
4. how about memory_values.py ? Also looks like memory APIs but not sure I understand this properly
5. What do we get from exposing the TABLE, TABLE_COLUMNS and NODE_TYPE in MemoryNode ?
6. We need to define the boundaries of SQL queries imo. Having raw sql queries in databse_access functions in memory makes sense, but some of these internal accessors are being called outside (https://github.com/facebookresearch/droidlet/blob/d96ec5b79014d44d01516fdff83ff0d32ce309ee/base_agent/loco_mc_agent.py#L319 , https://github.com/facebookresearch/droidlet/blob/cd30b44d0775695af15aa7209e7ab9a9e42b94f4/locobot/agent/loco_memory_nodes.py#L80 ) with raw SQL and in memory nodes.
7. We expose TABLE and TABLE_COLUMNS and NODE_TYPE - what's the purpose?
8. Right now we have both : memory.add_triple(subj=memid, pred_text="has_name", obj_text=mobtype) and memory.tag(memid, "VOXEL_OBJECT") Should we use one ?
9. We should replace untag in memory with delete_triple
10. An aside: we aren't quite using SetNode should we remove ?
11. Why is dances a property of mc_memory ?

[kavyasrinet]

For visualization: We need a high level diagram for the current state of memory, the schema, the architecture, the base class and children We need two things:

high-level understanding, Something on low level details for each agent.

Pointers from Soumith: Example diagram from graphviz: #337 Generated with: https://gist.github.com/soumith/9c6a9a21e426423f55207d2d291c7f7a and the command neato -T png architecture.dot -o out.png