Support for Referenced Objects

[subhashb]

Proposal:

Support relationships via the following two concepts:

Foreign Keys

• Indicated via the references option in Field definition
• validated during save() method call
• The value passed to references is the name of the attribute in the other entity (to be cross-referenced against)

Virtual Linkages

• Linked class defines a Virtual field, that is not persisted in the database and gives back the result object instance(s)
• via specifies the Linkage attribute, which can either be in the same entity (self-linkage) or a target entity
• many indicates multiple object instances, usually the "many" side of One-to-Many or Many-to-One relationships

The thought behind these concepts is to:

• Keep the relationship non-magical, meaning explicitly-defined
• Be able to adapt to different kinds of DBs (SQL, non-SQL, graph, etc.)
• Clearly distinguish between virtual in-memory objects (resulting from relationships) vs. the physical keys that will be present as part of the table

[subhashb]

Example:

class Account(Entity):
    posts = field.Linked(Post, via='Post.author_id', many=True)   # Virtual Field
    created_by = field.Integer(references='account.id')  # Self-referencing Foreign Key attribute
    updated_by = field.Integer(references='account.id')

class Post(Entity):
    comments = field.Linked(Comment, via='Comment.post_id', many=True)  # Virtual Field
    author = field.Linked(Author, via='created_by')  # Virtual Field, created on an attribute in the same entity
    created_by = field.Integer(references='account.id')
    updated_by = field.Integer(references='account.id')

class Comment(Entity):
    post_id = field.Integer(references='post.id')  # Foreign Key attribute
    post = field.Linked(Post, via='post_id')  # Virtual Field
    created_by = field.Integer(references='account.id')
    updated_by = field.Integer(references='account.id')

[subhashb]

Pros of automatic two-way field creation based on relationship definitions:

• Avoids duplicate field definitions
• Removes the need for validation of field on the "Other" Entity, like in case of the "Many" end of One-To-Many relationships
• Helps query from both sides automatically

Cons:

• Puts default assumptions on linkages (And an SQL database?)
• Creates fields by "Magic."

[subhashb]

Also, all references are lazy-loaded whenever accessed. We can probably give an eager_load option in the future.

[abhishek-ram]

I think your idea is good in the case of Linked field, but in the case of references its not clear what is happening. I was thinking of something like this:

class Account(Entity):
    posts = field.ReverseRelated(Post, key='created_by', many=True)   # Virtual Field
    created_by = field.Related('Account')  # Self-referencing Foreign Key attribute, will automatically create field called created_by_id
    updated_by = field.Related('Account')

class Post(Entity):
    comments = field.ReverseRelated(Comment, key='post', many=True)  # Virtual Field
    created_by = field.Related(Account) 
    updated_by = field.Related(Account) 

class Comment(Entity):
    post = field.Related(Post)  # Virtual Field
    created_by = field.Related(Account)
    updated_by = field.Related(Account)

[subhashb]

I think this is a good idea as well. For me, it distills the linkages even further and makes them even more explicit.

A few cons that come to mind:

• Linked via translates well to its intent than ReverseRelated key. The latter seems like technical jargon and incompatible with expressing domain driven design
• I wanted to avoid ReverseRelated and Related by simply denoting linkages, and the only purpose of linkages being easier data access than depicting relationship attributes accurately. Also, these terms announce the use of a Relationship-based medium (like SQL based databases), which may not always be the case.
• Related(<Entity>) presupposes creation of a Relation, but I was thinking about only a pure Foreign key with references. So created_by exists as part of Post whether it is linked or not; we can have an additional check by referencing its value to another Entity's records as a bonus.
• The magic creation of created_by_id is what I wanted to avoid by using an explicit reference. While it makes the schema structure more homogenous and conventional, I think it has a place in the database world and not in the Domain Logic world. When a schema is created to mirror the entity in a database, we can still map created_by to created_by_id, and both expressions would remain intact: created_by would stand for a person in the domain world (Post.created_by), and created_by_id would stand for the identifier in raw form.

What say you?

[subhashb]

Notes:

• It's pretty tricky (without introducing some weirdish code) to get one field to refer or shadow another field. An example would be having both post_id and post as fields, and having them point to the same target. This duplication seems redundant.
• Django's representation of entities, without having to worry about or exposing the underlying id column is probably a great way to emulate in our application too. We should talk in the language of objects and not identifiers.
• Having virtual fields (without a backing representation in the database) may end up causing issues while filtering with child attributes.

[subhashb]

A different approach, combining your input and further discovery:

class Account(Entity):
    username = field.String(max_length=50)
    password = field.String(max_length=255)
    last_logged_in = field.DateTime(default=datetime.utcnow)
    author = field.HasOne(Author)
    created_by = field.BelongsTo(Account)
    updated_by = field.BelongsTo(Account)

class Author(Entity):
    account = field.BelongsTo(Account)
    bio = field.Text()
    posts = field.HasMany(Post)

class Post(Entity):
    title = field.String(max_length=255)
    author = field.BelongsTo(Author)
    content = field.Text()
    comments = field.HasMany(Comment)
    tags = field.HasAndBelongsToMany(Tag, through=TagMap)
    created_at = field.DateTime(default=datetime.utcnow)
    updated_at = field.DateTime(default=datetime.utcnow)

class Comment(Entity):
    content = field.Text()
    post = field.BelongsTo(Post)
    created_at = field.DateTime(default=datetime.utcnow)

class Tag(Entity):
    name = field.String(max_length=50)
    posts = field.HasAndBelongsToMany(Post, through=TagMap)

class TagMap(Entity):
    tag = field.BelongsTo(Tag)
    post = field.BelongsTo(Post)
    valid_until = field.DateTime()

Example usage:

admin = Account.find_by(username='email')
account = Account.create(username='subhash', password='password', created_by=admin, updated_by=admin)
author = Author.create(account=account, bio='Blah Blah')
post1 = Post.create(title='Title Non-comprendo', author=author, content='Loren Ipsum...')
author.posts.create(title='Title Somewhat-comprendo', content='Loren Ipsum Redefined') # Single step save

tag1 = Tag.create(name='History')
tag2 = Tag.create(name='Sociology')
posts.tags.add(tag1, tag2)
posts.save() #  Two-step save

Notes:

• Auto-generated columns with <>_id are generated wherever applicable
• Backing <>_id fields are created on the right entity (the one that has BelongsTo defined in it and the entity with HasOne has a virtual field

Let me know what you think.