Slow while declaring in Loop

[Gauraviitkgp]

Hi, I've like 100,000 instaces of a class each which is calling the machine.

class Matter(object):
    pass

machine = Machine(states=states, transitions=transitions, initial='Healthy')

print("At End:",timeit.time.process_time())
for i in range(100000):
    lump = Matter()
    machine = Machine(lump, states=states, transitions=transitions, initial='Healthy')
print("At End:",timeit.time.process_time())

This process takes around 28 seconds to complete which is kinda slowing things up. Is there any faster way to do it? If we do by Machine.add_model() it takes

machine = Machine(states=states, transitions=transitions, initial='Healthy')
lump=[]
print("At End:",timeit.time.process_time())
for i in range(100000):
    lump.append(Matter())
print("At End:",timeit.time.process_time()) 
machine.add_model(lump)
print("At End:",timeit.time.process_time())

This Takes around 79 seconds. Any suggestions on how to speed up?

[noelwilsondel]

@Gauraviitkgp incase you haven't seen it there is some excellent information in this issue: https://github.com/pytransitions/transitions/issues/146

I'm currently trying to solve my performance issues using some of those techniques but am curious to know if this is a possible regression.

Thanks for this library I want to find a solution so we can use it as I think it's great.

[noelwilsondel]

I also took a trick from aleneum in #146 and if we change the models list to a set the computation time drastically decreases for your example, would it be possible to update this in the project (I'm happy to make a PR).

class Matter(object):
    pass

machine = Machine(states=states, transitions=transitions, initial='Healthy')

class SetWrapper(set):
    def append(self, item):
        self.add(item)

machine.models = SetWrapper()

lump=[]
print("At End:",timeit.time.process_time())
for i in range(100000):
    lump.append(Matter())
print("At End:",timeit.time.process_time()) 
machine.add_model(lump)
print("At End:",timeit.time.process_time())

[Gauraviitkgp]

Yes, it seems to fasten up things a lot, it's now nearly 5.23 seconds. Disabling auto transitions is reduces it to further 4.23 seconds which seems to reduce load massively. I'll update this to my project. Should I consider this as an official solution and close the issue?

[aleneum]

You might also want to have a look at the Frequently Asked Questions. What makes thinks slow is the dynamic decoration of models. If you do not need all the bells ans whistles have a look at transitions memory footprint is too large for my Django app and adding models takes too long. Unfortunately, anchors do not work well in Jupyter Notebooks so you might need to scroll a bit.

from transitions import Machine
from functools import partial
import timeit

class Model:

    machine = Machine(model=None, states=['A', 'B', 'C'], initial=None,
                      transitions=[
                          {'trigger': 'go', 'source': 'A', 'dest': 'B', 'before': 'before'},
                          {'trigger': 'check', 'source': 'B', 'dest': 'C', 'conditions': 'is_large'},
                      ])

    def __init__(self):
        self.state = 'A'

    @staticmethod
    def is_large(value=0):
        return value > 9000

    @staticmethod
    def before():
        print('before called')

    def __getattribute__(self, item):
        try:
            return super(Model, self).__getattribute__(item)
        except AttributeError:
            if item in self.machine.events:
                return partial(self.machine.events[item].trigger, self)
            raise

lump = []
start_time = timeit.time.process_time()
print("At Start:", start_time)
for i in range(100000):
    lump.append(Model())
end_time = timeit.time.process_time()
print("At End:", end_time)
print(f"Adding models took {end_time - start_time} seconds")
# testing triggers
lump[0].go()
assert lump[0].state == 'B'
assert lump[1].state == 'A'

On my notebook, this results in:

At Start: 0.859375
At End: 1.03125
Adding models took 0.171875 seconds
before called

So, the trick is to NOT add models to a machine and only use a class machine to handle all the transition logic. This way models are not decorated but can be triggered nevertheless. The downside is that you need to manage a collection of your models yourself and it lacks some convenience. However, if you are willing to extend __getattribute__, you can get some of it back. As already pointed out, a set can be used to increase lookup speed for your self-maintained models.

[noelwilsondel]

I'll have a go at implementing a similar solution to the example above, thanks for that @aleneum!

[Gauraviitkgp]

Hi thanks, @aleneum this works like a charm, sorry I thought the FAQ was specific to Django (maybe you can modify the heading a bit). Closing the issue Thanks