Optimal architecture for DHT record logic

[pospi]

This is a thread to discuss Rust's language features and how we best implement the DHT code... presuming Philip doesn't come along with the new GraphQL API generation feature and obviate us needing to hand-code most of the backend ;)

The first thing about Rust is that it's an ML-derived language and neither of us has learned any ML before. This will probably make the experience somewhat painful for a while until we attain some lightbulb moments, after which it will suddenly become amazing. It might be an idea to have weekly catch-ups where we compare notes on our learning as this will help accelerate each other. I will keep updating this thread as I learn new insights so that you can alert me if you've come to different conclusions.

Type hints:

Something I have seen in a couple of 'best practise' documents is to make type declarations "clear and expressive, without being cumbersome". However, information on how to make such distinctions is lacking.

From what I can tell, the compiler mandates that you declare a return type and parameter types for all functions. I suspect the above guideline is around the 'restrictiveness' of type parameters, and that the best practise is to make your types as generic as possible (eg. using &str instead of String to allow values which are not instances of the String class but can be converted to be used).

Record architecture:

The topic I've been debating lately is how best to architect the VF core functionality. Rust is not an OO language, and so preferring composition over inheritance is not only good practise here, it's also idiomatic and more performant, and AFAIK OO is not even really an option. Rust's trait system looks to be a really solid and strongly-typed way of dealing with mixins, though- we are in good hands.

The HDK methods for data handling are all quite low-level. Some amount of wrapping them up will be needed, especially around links. And then we need some higher-order wrapping that combines an entry and some links to create a record, like we did with GoChain. I imagine we will want very similar functionality.

The other challenge with the core VF fields is that we probably need to change our way of thinking, because a) you can't inherit structs, and b) traits cannot define fields. Rust really enforces that you keep your data separate from your behaviour. As a consequence, I suspect we will need to use macros to declare our entry types succinctly and avoid having to redeclare fields.

So this is what I came up with as a rough scratchpad for implementing a higher-order struct to manage all the related record data, and a trait implementation for managing it:

#[derive(Eq, PartialEq, Debug)]
enum LinkOrLinkList {
    Link(Address),
    LinkList(Vec<Address>),
}

#[derive(Eq, PartialEq, Debug, Default)]
pub struct Record<T> {
    entry_type: String,
    entry: T,
    address: Option<Address>,
    links: HashMap<String, LinkOrLinkList>,
}

trait LinkedRecord {
    fn commit(self);    // :TODO: return something useful
}

impl<T> LinkedRecord for Record<T> {
    fn commit(self) {
        // save entry
        let entry = Entry::App(self.entry_type.into(), self.entry.into());
        let address = hdk::commit_entry(&entry);
        match address {
            Err(e) => { /* :TODO: bail */ }
            Ok(a) => {
                self.address = Some(a);
            }
        }

        // save links
        for (tag, addr) in &self.links {
            match addr {
                LinkOrLinkList::Link(link) => {
                    match &self.address {
                        None => { /* should probably throw some error here, or check `address` above */ },
                        Some(self_addr) => {
                            // :TODO: handle result
                            link_entries(&self_addr, &link, tag.to_string());
                        }
                    }
                },
                LinkOrLinkList::LinkList(links) => {
                    match &self.address {
                        None => { /* ... */ },
                        Some(self_addr) => {
                            for link in links {
                                // :TODO: handle result
                                link_entries(&self_addr, &link, tag.to_string());
                            }
                        }
                    }
                }
            }
        }
    }
}

Some notes about this:

• There are probably a lot of Rust n00b misconceptions and mistakes in here.
• The first line of the commit function fails on self.entry.into(), and I couldn't figure out how to declare the type for the LinkedRecord implementation to make that work. I don't really understand the Into and From traits yet, they seem like dark magic.
• Gave some conditional logic a try with "single link or array of links" via the LinkOrLinkList enum. The way the language handles these felt overbearing at first, but I can see the benefits. It's impossible to miss a condition unless you explicitly want to.
• Error handling is probably going to be an interesting exercise. The docs on it go really deep.
• Of course all this needs to be split up and made to do all the cool things your TS LinkRepo does.
• Writing it took me a very long time. It's going to be a steep learning curve, but on the upside the type system is so good that you practically cannot write invalid code. The downside is how frequently the compiler barfs at you.

Anyway, that's my half-baked thoughts after a day and a half or so of learning Rust. If I'm doing stupid things or on the wrong track I would love to know about it! heh

[bhaugen]

@pospi do you think it is possible to differentiate ontology from taxonomy, or has that train left the station (if you get what I mean)?

I think they are distinctly different:

• an ontology is a set of relationships among concepts, where the relationships are mostly not hierarchical: like the basic REA ontology has no hierarchicall relationships;
• a taxonomy consists mostly of hierarchical relationships, like the https://en.wikipedia.org/wiki/Linnaean_taxonomy or http://aims.fao.org/vest-registry/vocabularies/agrovoc, where a granny smith apple is an apple is a pome fruit is a fruit etc...

People will use a variety of taxonomies with VF concepts like resource classification. Agrovoc will be one of them. Wikidata is another.

But I do understand that people have used the terms interchangeably.

[pospi]

Closing this- further iteration is covered by #22 & #60.