Problems with extending classes

[gotlium]

I have a several models, which I want to add to partitioning. I extend this models from abstract partitionable model and got follow exception:

AttributeError: 'Options' object has no attribute 'partition_column'

Code example for reproduce:

class PartitionableAbstract(Partitionable):
    class Meta:
        abstract = True

        partition_type = 'range'
        partition_subtype = 'date'
        partition_range = 'month'
        partition_column = 'date'

class User(PartitionableAbstract):
    date = models.DateField(db_index=True)
    num = models.PositiveIntegerField(default=0)

class Logs(PartitionableAbstract):
    date = models.DateField(db_index=True)
    num = models.PositiveIntegerField(default=0)

class Some(PartitionableAbstract):
    date = models.DateField(db_index=True)
    num = models.PositiveIntegerField(default=0)

[maxtepkeev]

I can't reproduce this error, here's what I did:

1) created empty app dbparti_test 2) copy'n'pasted your code into models.py 3) ran command python manage.py syncdb 4) ran command python manage.py partition dbparti_test 5) added the following code to admin.py to add models to the admin:

class UserAdmin(PartitionableAdmin):
    partition_show = 'all'

site.register(User, UserAdmin)

class LogsAdmin(PartitionableAdmin):
    partition_show = 'all'

site.register(Logs, UserAdmin)

class SomeAdmin(PartitionableAdmin):
    partition_show = 'all'

site.register(Some, UserAdmin)

6) successfully added new records in each model

Result: partition was created for each model

Are you doing something differently ? At what moment do you receive this error ? I need to know more to reproduce it. Thanks.

[gotlium]

on sync db. but on project I use south. I tried reproduce this problem by my self, and can not do it too. when this problem happen again, I reopen this issue, and after give you more information. sorry to this trouble.

[maxtepkeev]

I tried the same with South, but still can't reproduce the error, everything works as expected. If you will be able to find the steps to reproduce it again, please let me know. Thanks.

[gotlium]

can you tell me about how migration work with partition? I really want to know about any problems about alter table and partitioning in mysql 5.5.x. thanks

[gotlium]

Ok, what I did to reproduce this problem:

1. created app
2. initial schemamigration (./manage.py schemamigration app --initial)
3. migrating (./manage.py migrate app)
4. initial partitioning (./manage.py partition app)

so after I get following error:

Traceback (most recent call last):
  File "./manage.py", line 24, in <module>
    execute_manager(settings)
  File "/Users/gotlium/.env/test/lib/python2.7/site-packages/django/core/management/__init__.py", line 459, in execute_manager
    utility.execute()
  File "/Users/gotlium/.env/test/lib/python2.7/site-packages/django/core/management/__init__.py", line 382, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/Users/gotlium/.env/test/lib/python2.7/site-packages/django/core/management/base.py", line 196, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/Users/gotlium/.env/test/lib/python2.7/site-packages/django/core/management/base.py", line 232, in execute
    output = self.handle(*args, **options)
  File "/Users/gotlium/.env/test/lib/python2.7/site-packages/django/core/management/base.py", line 304, in handle
    app_output = self.handle_app(app, **options)
  File "/Users/gotlium/.env/test/lib/python2.7/site-packages/dbparti/management/commands/partition.py", line 18, in handle_app
    model_instance.get_partition().prepare()
  File "/Users/gotlium/.env/test/lib/python2.7/site-packages/dbparti/models.py", line 24, in get_partition
    current_value=self._meta.partition_column,
AttributeError: 'Options' object has no attribute 'partition_column'

Additional info:

# django-admin.py --version
1.4.2

# python --version
Python 2.7.5

# pip freeze |grep -i south
South==0.7.3

# pip freeze |grep -i mysql
MySQL-python==1.2.3

Can you check, please? But check with abstract class.

[maxtepkeev]

Sorry, still can't reproduce it ;-( Did the same steps as you, i.e.:

1. created app dbparti_test
2. initial schemamigration (./manage.py schemamigration dbparti_test --initial)
3. migrating (./manage.py migrate dbparti_test)
4. initial partitioning (./manage.py partition dbparti_test)

Result: Successfully (re)configured the database for the following models: User, Logs, Some

The code from models.py:

from django.db import models
from dbparti.models import Partitionable

class PartitionableAbstract(Partitionable):
    class Meta:
        abstract = True

        partition_type = 'range'
        partition_subtype = 'date'
        partition_range = 'month'
        partition_column = 'date'

class User(PartitionableAbstract):
    date = models.DateField(db_index=True)
    num = models.PositiveIntegerField(default=0)

class Logs(PartitionableAbstract):
    date = models.DateField(db_index=True)
    num = models.PositiveIntegerField(default=0)

class Some(PartitionableAbstract):
    date = models.DateField(db_index=True)
    num = models.PositiveIntegerField(default=0)

Environment:

Django==1.4.2
MySQL-python==1.2.3
South==0.7.3
wsgiref==0.1.2

It should work for you also, please check for .pyc/.pyo files and delete them all, restart your web server after this procedure and try once again.

As for your first question about migrations and partitioned tables - they work good together, e.g. if you add/remove a field / several fields, they all got added/removed from all partitions.

[gotlium]

Full example to reproduce:

from django.db import models
from dbparti.models import Partitionable
from mptt.models import MPTTModel, TreeForeignKey

class PartitionableAbstract(Partitionable):
    class Meta:
        abstract = True

        partition_type = 'range'
        partition_subtype = 'date'
        partition_range = 'month'
        partition_column = 'date'

class User(PartitionableAbstract):
    date = models.DateField(db_index=True)
    num = models.PositiveIntegerField(default=0)

class Logs(PartitionableAbstract):
    date = models.DateField(db_index=True)
    num = models.PositiveIntegerField(default=0)

class Some(MPTTModel, PartitionableAbstract):
    date = models.DateField(db_index=True)
    num = models.PositiveIntegerField(default=0)
    parent = TreeForeignKey(
        'self', null=True, blank=True, related_name='children')

    def __unicode__(self):
        return self.date

    class MPTTMeta:
        order_insertion_by = ['num']

Requirements:

Django==1.4.2
MySQL-python==1.2.5
Pillow==2.1.0
South==0.7.3
django-db-parti==0.3.1
django-mptt==0.6.0
ipython==1.1.0
mercurial==2.8.2
wsgiref==0.1.2

Steps:

(test)## ./manage.py schemamigration app --initial
Creating migrations directory at '/Users/gotlium/Projects/temp/test/app/migrations'...
Creating __init__.py in '/Users/gotlium/Projects/temp/test/app/migrations'...
 + Added model app.User
 + Added model app.Logs
 + Added model app.Some
Created 0001_initial.py. You can now apply this migration with: ./manage.py migrate app

(test)## ./manage.py migrate app
Running migrations for app:
 - Migrating forwards to 0001_initial.
 > app:0001_initial
/Users/gotlium/.env/test/lib/python2.7/site-packages/django/db/models/fields/__init__.py:808: RuntimeWarning: DateTimeField received a naive datetime (2014-02-07 10:40:28.774670) while time zone support is active.
  RuntimeWarning)
 - Loading initial data for app.
Installed 0 object(s) from 0 fixture(s)

(test)## ./manage.py partition app
Traceback (most recent call last):
  File "./manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/Users/gotlium/.env/test/lib/python2.7/site-packages/django/core/management/__init__.py", line 443, in execute_from_command_line
    utility.execute()
  File "/Users/gotlium/.env/test/lib/python2.7/site-packages/django/core/management/__init__.py", line 382, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/Users/gotlium/.env/test/lib/python2.7/site-packages/django/core/management/base.py", line 196, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/Users/gotlium/.env/test/lib/python2.7/site-packages/django/core/management/base.py", line 232, in execute
    output = self.handle(*args, **options)
  File "/Users/gotlium/.env/test/lib/python2.7/site-packages/django/core/management/base.py", line 304, in handle
    app_output = self.handle_app(app, **options)
  File "/Users/gotlium/.env/test/lib/python2.7/site-packages/dbparti/management/commands/partition.py", line 18, in handle_app
    model_instance.get_partition().prepare()
  File "/Users/gotlium/.env/test/lib/python2.7/site-packages/dbparti/models.py", line 24, in get_partition
    current_value=self._meta.partition_column,
AttributeError: 'Options' object has no attribute 'partition_column'

I think this problem happens when you add mptt.

[maxtepkeev]

Yes. This happenes because of this:

class Some(MPTTModel, PartitionableAbstract):

You are using multiple inheritance here and it works this way: if the attribute is found in the first class then it is not searched in the second and so on. MPTTModel has the Meta class and that means that Python is ignoring the Meta class in the PartitionableAbstract, that is why you are getting this exception.

That also means that if you do it this way:

class Some(PartitionableAbstract, MPTTModel):

django-db-parti will work but you will get other exception but now related to the django-mptt.

To make both libraries happy you should do it this way:

class Some(MPTTModel, PartitionableAbstract):
    date = models.DateField(db_index=True)
    num = models.PositiveIntegerField(default=0)
    parent = TreeForeignKey(
        'self', null=True, blank=True, related_name='children')

    def __unicode__(self):
        return self.date

    class Meta(PartitionableAbstract.Meta, MPTTModel.Meta):
        pass

    class MPTTMeta:
        order_insertion_by = ['num']

But this won't work either because django-mptt creates a foreign key and as I already said MySQL doesn't allow to partition tables with foreign keys which uses InnoDB.

The conclusion: to make everything work with this two libraries you will need to create your table using MyISAM engine.

[gotlium]

Thanks. But I need InnoDB. I create tables manually and that solved my problem.