SelfAttributes Fail When Called Through PostGeneration functions

[marky1991]

I have a simplified set of factory definitions below. If i try to create a WorkOrderKit object, via WorkOrderKitFactory(), it successfully generates a workorderkit with factory_boy 2.6.1 but fails with 2.9.2. I'm wondering if this is a bug or if it worked unintentionally before and this is the intended behavior. (If it is the intended behavior, do you have any suggestions on achieving this behavior now?)

The whole example django project: https://bitbucket.org/marky1991/factory-test/ .

If you would like to test it yourself, checkout the project, setup the database, run setup_db.psql, and then run factory_test/factory_test/factory_test_app/test.py.

Please let me know if anything is unclear or if you have any questions.

import factory
from factory.declarations import SubFactory, SelfAttribute
from factory.fuzzy import FuzzyText, FuzzyChoice
from factory_test_app import models

class ItemFactory(factory.DjangoModelFactory):
    class Meta:
        model = models.Item

    barcode = factory.fuzzy.FuzzyText(length=10)

class OrderHdrFactory(factory.DjangoModelFactory):
    order_nbr = factory.fuzzy.FuzzyText(length=20)
    class Meta:
        model = models.OrderHdr

    @factory.post_generation
    def order_dtls(self, create, extracted, **kwargs):
        if not create:
            return

        if extracted is not None:
            for order_dtl in extracted:
                order_dtl.order = self
                author.save()
            return
        for _ in range(5):
            OrderDtlFactory(order=self,
                            **kwargs)

class WorkOrderKitFactory(factory.DjangoModelFactory):
    class Meta:
        model = models.WorkOrderKit

    work_order_nbr = factory.fuzzy.FuzzyText(length=20)
    item = SubFactory(ItemFactory)
    sales_order = SubFactory(OrderHdrFactory,
                             order_dtls__item=SelfAttribute("..item"))

class OrderDtlFactory(factory.DjangoModelFactory):
    class Meta:
        model = models.OrderDtl

    order = SubFactory(OrderHdrFactory,
                       order_dtls=[])
    item = SubFactory(ItemFactory)

The traceback in 2.9.1:

Traceback (most recent call last):
  File "factory_test_app/test.py", line 8, in <module>
    kit = WorkOrderKitFactory()
  File "/home/lgfdev/ve_factory_test/local/lib/python2.7/site-packages/factory/base.py", line 46, in __call__
    return cls.create(**kwargs)
  File "/home/lgfdev/ve_factory_test/local/lib/python2.7/site-packages/factory/base.py", line 568, in create
    return cls._generate(enums.CREATE_STRATEGY, kwargs)
  File "/home/lgfdev/ve_factory_test/local/lib/python2.7/site-packages/factory/base.py", line 505, in _generate
    return step.build()
  File "/home/lgfdev/ve_factory_test/local/lib/python2.7/site-packages/factory/builder.py", line 275, in build
    step.resolve(pre)
  File "/home/lgfdev/ve_factory_test/local/lib/python2.7/site-packages/factory/builder.py", line 224, in resolve
    self.attributes[field_name] = getattr(self.stub, field_name)
  File "/home/lgfdev/ve_factory_test/local/lib/python2.7/site-packages/factory/builder.py", line 366, in __getattr__
    extra=declaration.context,
  File "/home/lgfdev/ve_factory_test/local/lib/python2.7/site-packages/factory/declarations.py", line 306, in evaluate
    return self.generate(step, defaults)
  File "/home/lgfdev/ve_factory_test/local/lib/python2.7/site-packages/factory/declarations.py", line 395, in generate
    return step.recurse(subfactory, params, force_sequence=force_sequence)
  File "/home/lgfdev/ve_factory_test/local/lib/python2.7/site-packages/factory/builder.py", line 236, in recurse
    return builder.build(parent_step=self, force_sequence=force_sequence)
  File "/home/lgfdev/ve_factory_test/local/lib/python2.7/site-packages/factory/builder.py", line 296, in build
    context=postgen_context,
  File "/home/lgfdev/ve_factory_test/local/lib/python2.7/site-packages/factory/declarations.py", line 570, in call
    instance, create, context.value, **context.extra)
  File "/home/lgfdev/factory_test/factory_test/factory_test_app/factories.py", line 29, in order_dtls
    **kwargs)
  File "/home/lgfdev/ve_factory_test/local/lib/python2.7/site-packages/factory/base.py", line 46, in __call__
    return cls.create(**kwargs)
  File "/home/lgfdev/ve_factory_test/local/lib/python2.7/site-packages/factory/base.py", line 568, in create
    return cls._generate(enums.CREATE_STRATEGY, kwargs)
  File "/home/lgfdev/ve_factory_test/local/lib/python2.7/site-packages/factory/base.py", line 505, in _generate
    return step.build()
  File "/home/lgfdev/ve_factory_test/local/lib/python2.7/site-packages/factory/builder.py", line 275, in build
    step.resolve(pre)
  File "/home/lgfdev/ve_factory_test/local/lib/python2.7/site-packages/factory/builder.py", line 224, in resolve
    self.attributes[field_name] = getattr(self.stub, field_name)
  File "/home/lgfdev/ve_factory_test/local/lib/python2.7/site-packages/factory/builder.py", line 366, in __getattr__
    extra=declaration.context,
  File "/home/lgfdev/ve_factory_test/local/lib/python2.7/site-packages/factory/declarations.py", line 137, in evaluate
    target = step.chain[self.depth - 1]
IndexError: tuple index out of range

[flaeppe]

I've run into a similar issue when upgrading to >=2.9.0. Although my issue appeared when utilising factory.Trait together with post_generation which seemed to have some internal implementation changed to now use factory.SelfAttribute(this is my suspicion at least).

This problem sounds to (might) have been introduced in my case when:

Add :class:factory.Maybe, which chooses among two possible declarations based on another field's value (powers the :class:~factory.Trait feature).

I'm currently using 2.7.0, where the following snippet passes(also on 2.8.x versions), it fails however on >=2.9.0:

I'm also running Python 3.5 if that would happen to make any difference.

import factory
from django.test import TestCase

class MyGroupFactory(factory.django.DjangoModelFactory):

    class Meta:
        model = 'auth.Group'
        django_get_or_create = ('name',)

    class Params:
        admin = factory.Trait(name='admins')

class MyUserFactory(factory.django.DjangoModelFactory):
    email = factory.Sequence(lambda n: '%03d@email.com' % n)

    class Meta:
        model = 'core.UserProfile'

    class Params:
        admin = factory.Trait(groups__admin=True)

    @factory.post_generation
    def groups(self, create, extracted, **kwargs):
        if not create:
            return

        if extracted:
            self.groups.add(*extracted)
        elif kwargs:
            self.groups.add(MyGroupFactory(**kwargs))

class UserTests(TestCase):
    def test_failing_factory(self):
        MyUserFactory(admin=True)

I get a very similar traceback to @marky1991, hence I suspect that we've encountered the same bug here.

Traceback with >=2.9.0:

Traceback (most recent call last):
  File "/app/tests.py", line 43, in test_failing_factory
    MyUserFactory(admin=True)
  File "/usr/local/lib/python3.5/site-packages/factory/base.py", line 46, in __call__
    return cls.create(**kwargs)
  File "/usr/local/lib/python3.5/site-packages/factory/base.py", line 568, in create
    return cls._generate(enums.CREATE_STRATEGY, kwargs)
  File "/usr/local/lib/python3.5/site-packages/factory/base.py", line 505, in _generate
    return step.build()
  File "/usr/local/lib/python3.5/site-packages/factory/builder.py", line 296, in build
    context=postgen_context,
  File "/usr/local/lib/python3.5/site-packages/factory/declarations.py", line 570, in call
    instance, create, context.value, **context.extra)
  File "/app/tests.py", line 38, in groups
    self.groups.add(MyGroupFactory(**kwargs))
  File "/usr/local/lib/python3.5/site-packages/factory/base.py", line 46, in __call__
    return cls.create(**kwargs)
  File "/usr/local/lib/python3.5/site-packages/factory/base.py", line 568, in create
    return cls._generate(enums.CREATE_STRATEGY, kwargs)
  File "/usr/local/lib/python3.5/site-packages/factory/base.py", line 505, in _generate
    return step.build()
  File "/usr/local/lib/python3.5/site-packages/factory/builder.py", line 275, in build
    step.resolve(pre)
  File "/usr/local/lib/python3.5/site-packages/factory/builder.py", line 224, in resolve
    self.attributes[field_name] = getattr(self.stub, field_name)
  File "/usr/local/lib/python3.5/site-packages/factory/builder.py", line 366, in __getattr__
    extra=declaration.context,
  File "/usr/local/lib/python3.5/site-packages/factory/declarations.py", line 440, in evaluate
    choice = self.decider.evaluate(instance=instance, step=step, extra={})
  File "/usr/local/lib/python3.5/site-packages/factory/declarations.py", line 142, in evaluate
    return deepgetattr(target, self.attribute_name, self.default)
  File "/usr/local/lib/python3.5/site-packages/factory/declarations.py", line 104, in deepgetattr
    return getattr(obj, name)
  File "/usr/local/lib/python3.5/site-packages/factory/builder.py", line 366, in __getattr__
    extra=declaration.context,
  File "/usr/local/lib/python3.5/site-packages/factory/declarations.py", line 440, in evaluate
    choice = self.decider.evaluate(instance=instance, step=step, extra={})
  File "/usr/local/lib/python3.5/site-packages/factory/declarations.py", line 137, in evaluate
    target = step.chain[self.depth - 1]
IndexError: tuple index out of range

[marky1991]

@rbarrois Do you have any comments on the intended design? I'd be willing to work on this issue if I knew what, if anything, you want to do about it. (but if nothing, what workaround you suggest)

[rbarrois]

Hey, sorry for the awfully late reply :/

I think this was actually the same issue as #466 — which I just fixed in the 2.11.0 release.

Could you check if the issue still occurs for you?

[flaeppe]

@rbarrois running with >=2.11.0 the issue is resolved. 👍 (at least for my case above)

[Subaku]

Would like to bump this issue as not having been resolved just yet. I'm running the latest released version 2.11.1 myself. Here's some minimal code you can run to see the issue:

class Foo(object):
    thing = 1

    def __init__(self, thing):
        self.thing = thing

class Boo(object):
    foo = None
    thing2 = 9

    def __init__(self, foo, thing2):
        self.foo = foo
        self.thing2 = thing2

class FooFactory(f.Factory):
    thing = 5

    class Meta:
        model = Foo

    @f.post_generation
    def install(self, create, extracted, **kwargs):
        print 'post gen install'

class BooFactory(f.Factory):
    thing2 = 10
    foo = f.SubFactory(FooFactory, install=f.SelfAttribute('..thing2'))

    class Meta:
        model = Boo

If I try BooFactory() in one of my tests I get the following AttributeError:

AttributeError: The parameter 'thing2' is unknown. Evaluated attributes are {'thing': 5}, definitions are <DeclarationSet: {'thing': 5}>.

Does anyone have a workaround or an idea as to what's causing this?

Thanks!

[AlecRosenbaum]

I'm running into the same exact issue that @Subaku described above. Is there a workaround?

Edit: @Subaku's code should work in version factory-boy==2.8.1

[rbarrois]

@AlecRosenbaum could you share some code on the kind of issue you're seeing?

In @Subaku example code, what happens is that the overridden install value is designed to be passed into the extracted field of the post_generation method; and resolved within the context of that method's parameters.

In other words, there is an intermediate level where parameters for def install() are computed (required for cases where more parameters are provided to that declaration, where each parameter might use values from other parameters). When calling SelfAttribute('..thing2'), the .. part goes up a level to FooFactory, not to BooFactory; using factory.SelfAttribute('...thing2') should work.

However, passing in a pre-generation declaration (SelfAttribute here) to shadow a post-generation declaration makes the factories much harder to read, and could fail in unexpected ways; I recommend designing the factory differently, for instance playing with class Params for passing in parameters.

[AlecRosenbaum]

@rbarrois Huh, ok so adding an extra . on the SelfAttribute works starting in version 2.11.0 but doesn't work in lower versions. Thanks!

The situation I am setting up is a little different from the previous example, so I'll put some runnable code (with pytest) here documenting the use case. I'm not sure how I would use Params, since the code has to run after creation. Please let me know if I'm wrong about Params or if you have any suggestions on how to set it up cleaner.

Example Scenario

```python import factory # ============== # --- Models --- # ============== RELATIONSHIPS = set() def rel_key(_from, to, type): return "{}::{}::{}".format(_from, to, type) class User(object): def __init__(self, name): self.name = name def add_to_organization(self, org): # some mumbo-jumbo here RELATIONSHIPS.add(rel_key(_from=self.name, to=org.name, type="member")) def member_of_organization(self, org): key = rel_key(_from=self.name, to=org.name, type="member") return key in RELATIONSHIPS class Organization(object): def __init__(self, name): self.name = name class Event(object): def __init__(self, organization, user): self.organization = organization self.user = user # ================= # --- Factories --- # ================= class UserFactory(factory.Factory): name = "Johnny Smith" class Meta: model = User @factory.post_generation def organization(user_inst, create, extracted, **kwargs): if create and extracted: user_inst.add_to_organization(extracted) class OrganizationFactory(factory.Factory): name = "The Smith Organization" class Meta: model = Organization class EventFactory(factory.Factory): organization = OrganizationFactory() user = factory.SubFactory( UserFactory, organization=factory.SelfAttribute("...organization") ) class Meta: model = Event # ==================== # --- Example Test --- # ==================== def test_event(): my_event = EventFactory() assert my_event.organization assert my_event.user assert my_event.user.member_of_organization(my_event.organization) ```