Very slow AIG walking

[masinag]

Hi, I am using py-aiger to perform some AIG manipulation. In particular, I am trying to convert AIG to PySMT formulas. I have found an example of AIG waking in https://github.com/mvcisback/py-aiger/blob/f50171549781e0910cd5aec25dd8582a9d219b84/aiger/writer.py#L41

I have tried to adapt this to my scenario, but I have noticed this gets very slow on medium-big instances. And I mean very slow, like hours instead of seconds. E.g. https://github.com/yogevshalmon/allsat-circuits/blob/b57c2d6cba244460008dc6400beef2604a720c24/benchmarks/random_aig/large_cir_or/bench1/bench1.aag

It seems to me that the bottleneck is somewhere in aiger.common.dfs function, likely operations on sets of nodes. I suppose that this can be due to the computation of hash for nodes (generated by @attr.frozen), which traverses the whole subgraph each time, for each node.

I attach code to replicate the issue.

import aiger
import funcy as fn

def gates(circ):
    gg = []
    count = 0

    class NodeAlg:
        def __init__(self, lit: int):
            self.lit = lit

        @fn.memoize
        def __and__(self, other):
            nonlocal count
            nonlocal gg
            count += 1
            new = NodeAlg(count << 1)
            right, left = sorted([self.lit, other.lit])
            gg.append((new.lit, left, right))
            return new

        @fn.memoize
        def __invert__(self):
            return NodeAlg(self.lit ^ 1)

    def lift(obj) -> NodeAlg:
        if isinstance(obj, bool):
            return NodeAlg(int(obj))
        elif isinstance(obj, NodeAlg):
            return obj
        raise NotImplementedError

    start = 1
    inputs = {k: NodeAlg(i << 1) for i, k in enumerate(sorted(circ.inputs), start)}
    count += len(inputs)

    omap, _ = circ(inputs=inputs, lift=lift)

    return gg

def main():
    circ = aiger.to_aig("bench1.aag")
    gg = gates(circ)

    print(len(gg))

if __name__ == '__main__':
    main()

[mvcisback]

Hi @masinag ,

Thanks for reaching out. I can take a look sometime in the coming weeks.

I suspect you're right about the hash issue, it's been a bit of a wart for a while and one of the reasons the lazy API was initially developed -- although that won't help here.

I recommend looking at it with a tool like pyspy to get a flame graph is probably going to good to confirm.

https://github.com/benfred/py-spy

If you have a chance to take a look at the py-spy let me know (feel free to attach the output svg).

Supposing it is the hashing in common.dfs we can look at two solutions:

1. accelerating hashing in general.
2. re-writing common.dfs to avoid the hashing.

Option 1

For option 1, I would have thought this was solved by cache_hash.

https://github.com/mvcisback/py-aiger/blob/f50171549781e0910cd5aec25dd8582a9d219b84/aiger/aig.py#L53

Perhaps we're having a lot of hash collisions and being killed by equality checks? Eitherway it's strange worst case we'll need to manually introduce smarter hashing and caching.

Option 2

I think this is the easiest to code, but not a very satisfying solution. Essentially would could switch to checking if that exact node has already been emitted. This would be done perhaps as follows.

def dfs(circ):
    """Generates nodes via depth first traversal in pre-order."""
    emitted: set()
    stack = list(circ.cones | circ.latch_cones)

    while stack:
        node = stack.pop()

        if id(node) in emitted:
            continue

        remaining = [c for c in node.children if id(c) not in emitted]

        if len(remaining) == 0:
            yield node
            emitted.add(id(node))   # node -> id(node)
            continue

        stack.append(node)  # Add to emit after remaining children.
        stack.extend(remaining)

[mvcisback]

@masinag looking at your code again, it may actually be that NodeAlg doesn't cache its hashes. Could you try again with that?

[masinag]

@masinag looking at your code again, it may actually be that NodeAlg doesn't cache its hashes. Could you try again with that?

I don't think I understand what you mean. I don't see where I am hashing NodeAlg objects

[masinag]

About the proposed options, I can try to profile the execution with py-spy.

Option 2 seems an easy fix, but if the issue is really the hashing, speeding it up could improve performance in many other contexts. So it could be worth looking deeper into that.

[mvcisback]

@masinag looking at your code again, it may actually be that NodeAlg doesn't cache its hashes. Could you try again with that?

I don't think I understand what you mean. I don't see where I am hashing NodeAlg objects

Err, actually ignore what I said.