betamoo commented 6 years ago

Last Modified: 01/25/2018

Summary

PynamoDB currently doesn’t -guarantee to- support inheritance. We aim at supporting two different types of inheritance in Pynamo: Abstract & Proxy inheritance (explained later) in order to allow more flexible ways of designing/describing models for Pynamo users.

Goal

Support two types of inheritance Abstract & Proxy inheritance.

Non-goal: supporting Multi table inheritance.

Context

PynamoDB allows describing, creating and accessing tables in DynamoDB through pythonic styled methods. Creating tables in Pynamo is -mostly- declarative: in that by declaring python classes, the user can create & get access to DynamoDB tables and data.

Model inheritance is a feature that is supported by some ORMs, such as: Django

The three types of inheritance supported in Django:

1. Abstract Inheritance In which, the parent (inherited model) is an abstract model that doesn’t correspond to a table, but rather just a definition or an information holder for the children (inheriting models) to follow.

A use case of abstract inheritance is when more than one model share common field descriptions, the user can have an abstract class with the common fields and inherit it by different children, that way the user saves work that that otherwise would be redundant.

2. Proxy Inheritance In this case, the inheriting child (or children) provides) only an alternate way to access the same table of the parent model. Both the proxies and their parent correspond to the same table but can have different configurations or different behaviours.

A use case for proxy inheritance is when a user needs to access the same table with two or more different configurations (different port or different RCU/WCU). Another use case is to mask certain fields in the child proxy such that it only has access to a subset of it’s parent fields.

3. Multi table inheritance In which both the parent and each of the children represent different tables, but with links between the common fields and attributes. That means that some fields of corresponding models are mirrored in the different tables and have to stay in sync (including for example cascade delete).

A use case for multi table inheritance, is when the user needs to create a table that shadows the original table. The inheritance relation should take care of the data replication.

Plan

As mentioned earlier, we are aiming only to support abstract inheritance and proxy inheritance.

We are going to skip the multi table inheritance (at least for now) as it is almost impossible/will introduce many complications in a non relational key-value database like DynamoDB.

We are planning to approach the inheritance behaviour as follows:

Abstract Inheritance Behaviour

Abstract models can not be instantiated, and cannot be used to access data
Abstract models don’t correspond to tables
Abstract models can not have LSIs nor GSIs
- We could allow a form of abstract LSIs & abstract GSIs, that only can be used in abstract models. In that case, the indexes need to be redeclared in the non abstract children.
Abstract child can inherit an abstract parent
The Meta information of an abstract model is empty or trivial
- The Meta of an abstract model can not have a table name
By default, inheriting an abstract parent results in an abstract child (unless explicitly declared as non abstract)
Possible feature: when table_name is declared on models inheriting abstract models, automatically become non-abstract ?
The proposed way of declaring a model abstract is:
- The Meta of the model will define abstract = True

Proxy Inheritance Behaviour

Proxy models can not inherit models (neither of them correspond to actual tables)
Proxy models can only inherit:
- Models that correspond to actual tables (non-abstract models)
- Other proxy models.
Proxy models will correspond to their parent table.
Proxy models can access the parent’s LSIs and GSIs
The Meta class of a proxy model can have different information than the meta of the parent, except for the table_name field.
Proxy models can not declare their own fields (because they are only a proxy)
- Exception should be thrown in this case
- (Evaluate) Unless we choose to support proxy as a form of discriminated unions
By default, inheriting a non-abstract model, is a proxy inheritance
Proxy could be able to mask certain inherited fields
- (Figure out) syntax for this
Proxy can’t define hash/range/index
The proposed way of declaring a model abstract is:
- The Meta of the model will define proxy = True

General Notes & Behaviour

Instantiating children doesn’t automatically instantiate parents (valid for abstract & proxy inheritance)
Redeclaring fields are not allowed (valid for both abstract & proxy inheritance)
- (Evaluate) An exception will be for AbstractLSI and AbstractGSI if we decide to provide them
A child can override the metaclass inherited from a parent and change it's fields
Lateral inheritance will not be supported (inheriting from more than one direct parent)
Meta classes with different table_name correspond to different tables
We need to ensure (or declare the opposite) backward compatibility

Questions yet to be answered:

Should we enforce using/inheriting from one of the provided Meta classes for all Pynamo models?
Should we allow redundant declaration of the same attribute? (Possibly not)
Do we need to keep track internally of inherited attributes? (e.g. in a class dictionary inside a model)
What will the behaviour of inherited attributes that have alternative names?
Should non abstract models enforce having tabel_name?

Sample Code

Abstract Inheritance Example:

class UserModelBase1(Model):

    class Meta:
        abstract = True

    user_name = UnicodeAttribute(hash_key=True)
    email = UnicodeAttribute()

class UserModelBase2(UserModelBase1):
    custom_aliases = UnicodeSetAttribute(attr_name=**'aliases'**)
    views = NumberAttribute(null=True)

class UserModelBase3(UserModelBase2):
    icons = BinarySetAttribute()

class UserModel(UserModelBase3):

    class Meta:
        table_name = 'UserModel'
        abstract = False

    numbers = NumberSetAttribute()
    is_active = BooleanAttribute(null=True)
    signature = UnicodeAttribute(null=True)

UserModel.create_table(read_capacity_units=2, write_capacity_units=2)

Proxy Inheritance Example:


class UserModel(Model):

     class Meta:
         table_name = 'UserModel'
         abstract = False

    user_name = UnicodeAttribute(hash_key=True)
    email = UnicodeAttribute()
    icons = BinarySetAttribute()
    numbers = NumberSetAttribute()
    is_active = BooleanAttribute(null=True)
    signature = UnicodeAttribute(null=True)

class UserModelProxy1(UserModel):

     class Meta:
         proxy = True

class UserModelProxy2(UserModel):

     class Meta:
         read_capacity_units = 2
         write_capacity_units = 2
         proxy = True

Testing strategy

Abstract Testing

Multi level abstract model inheritance
A model with no Meta should default to using AbstractMeta
Inheriting abstract model defaults to an abstract child
An abstract model (either by definition or by inheritance) can be inherited into a non abstract model.
A non abstract child model inherited from an abstract model, should be virtually equivalent (in representation & in behaviour) to a ‘raw’ model that declares both attributes from the parent and the child
AbstractMeta throws exception if a table_name is provided, and suggests to use the normal Meta instead
Instantiating an abstract model throws exception
Abstract models with LSI or GSI throws exception
Non abstract children should be able to declare LSI and GSI on inherited fields
Redeclaring inherited fields throws exception
Changing a non abstract model to abstract is not allowed

Proxy Testing

Multi level proxy inheritance.
Inheriting a non-abstract model defaults to proxy inheritance
A proxy model inherited from the parent, should be able to behave virtually identical to the parent, except for the introduced behavioural changes.
Different Meta info does reflect in a different proxy behaviour
Data changes using proxy, should be visible through the parent, and vise versa
Proxy models throws exception if different table name is provided
Instantiating a proxy model whose parent isn’t instantiated throws exception
Proxy models with LSI or GSI throws exception
Redeclaring inherited fields throws exception

Abstract/Proxy Testing

Abstract model can’t inherit a proxy model
A proxy model can’t inherit an abstract model
Directly inheriting from two or more parents isn’t allowed
Already existing unit tests should work with:
A model that inherited from an abstract model
A proxy model that inherits the original model
A proxy model that inherits a non abstract model that inherits an abstract model

Related Tickets

Proposed PR

PR for abstract inheritance changes: https://github.com/pynamodb/PynamoDB/pull/440

the PR for proxy inheritance is coming soon

Jamim commented 6 years ago

Hi everyone, Are there any updates?