maxtepkeev / architect

A set of tools which enhances ORMs written in Python with more features
Other
391 stars 57 forks source link

Partitioning Feature Produces No Partitions #39

Closed ztroop closed 7 years ago

ztroop commented 7 years ago

Hi Max,

First of all, I'm excited to see that this project exists. I believe there's something missing in either the documentation or in my understanding of partitioning.

When I go to use this model, should this not have already installed at the database level? All of my data is being written to only one table, meaning partitioning doesn't seem to be working.

Any insights would be appreciated. I am likely just missing a step.



from sqlalchemy import Column, Integer, String
from sqlalchemy.ext.declarative import declarative_base

from project.app import example_engine

Base = declarative_base()

@architect.install(
    'partition', type='range', subtype='string_lastchars',
    constraint='2', column='example_field',
    db='postgresql+psycopg2://{}:{}@{}:{}/{}'.format( 'omitted', 'omitted', 'omitted', 'omitted', 'omitted')
)
class SomeModel(Base):
    __tablename__ = 'some_table_name'
    __table_args__ = (Index('_field', 'example_field', 'another_example_field'), )

    id = Column('id', Integer, primary_key=True)
    example_field = Column('example_field', String(32))
    another_example_field = Column('another_example_field', String(32))

Base.metadata.create_all(example_engine)```
ztroop commented 7 years ago

I'm just thick-witted, I just overlooked this as I thought it was Django-specific.

After the feature has been installed into the model, run the following console command: architect partition --module path.to.the.model.module

maxtepkeev commented 7 years ago

Hi Zackary! Yeah, I should probably restructure this part of documentation somehow to make it more understandable, because you're not the first one who missed that step and I'm afraid not the last one. Anyway, I'm glad that you figured it out.

robertour commented 5 years ago

Is there a way to call it directly from the code.

import architect
architect.apply(app.models)
robertour commented 5 years ago

Sorry, I just noticed that it is also in the documentation, somehow I unconsciously tend to blend the Notes out (probably because it usually clarify special cases). IMHO, there should not be notes; there should be part of the white text.

https://architect.readthedocs.io/features/partition/index.html

robertour commented 5 years ago

For Python 3, the parenthesis in the print are missing in the create_partition

def create_partitions(sender, **kwargs):
    """
    After running migrations, go through each of the models
    in the app and ensure the partitions have been setup
    """
    paths = {model.__module__ for model in sender.get_models()}
    for path in paths:
        try:
            partition.run(dict(module=path))
        except ProgrammingError:
            # Possibly because models were just un-migrated or
            # fields have been changed that effect Architect
            print("Unable to apply partitions for module '{}'".format(path))
        else:
            print("Applied partitions for module '{}'".format(path))