Organizing attractors

[jcrozum]

We need a better way to store attractors. I outline a tentative proposal below. Please let me know what you all think. For example, let me know if there's any information about attractors that I haven't mentioned that you think should be readily accessible. Conversely, let me know if you think this scheme would lead to "information overload" for the user. Also please speak up if you think there's a better approach to organizing the data; this is far from set in stone.

A class called AttractorRepertoire that contains objects of type AttractorSpace. An AttractorSpace would contain the following data:

• The set of nodes logically fixed (i.e., those that are part of a stable module
• All MotifReduction objects that have the same set of logically fixed nodes
• Upper and lower bounds for the number of attractors contained in the space
• If contained complex attractors have been explored . . .
  • The set of nodes that oscillate in each of those attractors
  • The complete STG subgraph of each attractor
• If contained complex attractors have not been explored or we don't know for sure if they exist . . .
  • The conditions for the existence of these attractors (this is the rspace constraint; it will be a tautology if we know with certainty that there is an attractor here)
  • If a reduction was computed, the complex attractors of the reduction as a list of oscillating nodes
  • If a reduction was computed, the complex attractors of the reduction as an STG subgraph

The AttractorRepertoire object would contain the following data:

• The original rules of the system
• A list of all AttractorSpace objects for the system
• A list of all steady states
• A list of all identified complex attractors
• A list of all AttractorSpaces that might (or are known to) contain unexplored attractors
• Upper and lower bounds on the number of complex attractors in total (i.e., across all attractor spaces)

The AttractorRepertoire object would also have member functions that summarize the data in various formats. Among these would be csv, json, and console output. There may be other formats (e.g., potentially pandas).

The SuccessionDiagram object could have an AttractorRepertoire variable, or it could be the other way around. Or it could even be completely separate. Maybe it should take the rules as input? Suggestions for how the constructor should work are especially welcome.

[deriteidavid]

This is a good idea. My thoughts:

The set of nodes logically fixed (i.e., those that are part of a stable module

How is this different from a steady-state? By the fact that it doesn't have all nodes fixed, i.e. some of them oscillate? We should come up with a name for these I guess. Something to do with "stable modules" would be nice.

All MotifReduction objects that have the same set of logically fixed nodes

This could be redundant given that this is an attractor class

The complete STG subgraph of each attractor

We should not store this in the memory, but it could be a generator object or function that returns it when called. We talked about this but I would also give some sort of heuristic option that samples the complex attractor STG if it's deemed too big. If you agree I will think of a way that doesn't bring in new dependencies.

The SuccessionDiagram object could have an AttractorRepertoire variable, or it could be the other way around. Or it could even be completely separate. Maybe it should take the rules as input? Suggestions for how the constructor should work are especially welcome.

I imagine a one-way connection. The succession diagram should have the attractor repertoire, but the other way around not necessarily. This depends on how we find the attractors in the first place. If a user can generate the attractor repertoire without building the succession diagram, they should be able to do that (I know PyBoolNet finds the steady states quite quickly). If we generate the succession diagrams anyway to have the attractor repertoire (it could be costly) then the two-way link makes more sense. Treating them completely separate is not very useful, because we want the attractors to be able to interpret the succession diagram, no?

Let me know what you think.

[jcrozum]

How is this different from a steady-state?

As you guessed, not all nodes need to be fixed. Stable module is the correct term, I think. These AttractorSpace objects would describe the system you get after plugging in a stable module (provided the resulting system has/might have motif-free attractors).

This could be redundant

In the PS, you might want to know that SAC contains p0->p2 in addition to just p2, even though LDOI(p2) = LDOI(p0+p2). There might be a way to compress this information though. The motif histories are stored in the MotifReduction object. Maybe we pick a representative and store a list of equivalent motif histories or MotifReductions?

This actually gives me an idea for speeding things up. I'll make a separate issue for that though.

We should not store this in memory

I am not so sure about this. It can take a very long time to generate the STG and we are just interested in some of its subgraphs. In practice, these are usually not too large or too numerous. We often need to build a partial STG when doing the reduction anyway, and if so, we save it in the MotifReduction object. If we store it as a generator, we will inevitably build it twice.

I'm not exactly sure how Python handles the memory in the case of a list shared by different classes, but presumably we can make these pointers to avoid copying the data.

heuristic option that samples the complex attractor

Yes, this would be nice, but it is low priority. I think it is fine to focus on exact methods until initial release.

The succession diagram should have the attractor repertoire, but the other way around not necessarily.

Makes sense.

This depends on how we find the attractors in the first place.

I am pretty sure our succession approach is almost always faster than PyBoolNet alone. But if you have a counterexample, let me know. I think we can proceed under the assumption that this attractor repertoire object would be independent of PyBoolNet's attractor finder. That means we are creating a succession diagram anyway.

Maybe we can modify the build_succession_diagram function so that it first does its usual thing, then it uses the succession diagram it constructed to compute an attractor repertoire that would be a member variable of the SuccessionDiagram class (this could be done using a member function of the SuccessionDiagram class). The second step would be comparatively cheap.

In that scheme, we would have Succession.py depend on Reduction.py and Attractors.py, while Attractors.py would only depend on Reduction.py.

[jgtz]

I like what you propose, Jordan.

In terms of the Attractors, Succession dependency. I like what you propose in the last post:

"Succession.py depend on Reduction.py and Attractors.py, while Attractors.py would only depend on Reduction.py."

One other option I was thinking on is to build a "master" class called "NetworkModel" or something of the form. This class would store a single AttractorRepertoire and SuccessionDiagram object. The creation of these objects could be called from the NetworkModel class. The advantage I can see is that:

• This could allow the AttractorRepertoire and SuccessionDiagram objects to remain separate. Their point of contact would be the "NetworkModel" class.
• This could help with the ""information overload" for a user. For a user that doesn't care at all about how we do things under the hood, it would allow the user to just "find attractors". Under the hood we could do the same thing we are doing (SuccessionDiagrams) and even implement other options (full STG or STG sampling, even if low priority now, vanilla PyBoolNet,etc).
• Of note, this is actually the way my java code is written (or, at least, what the original idea was).

Let me know what you think.

[jcrozum]

I guess I was thinking that the SuccessionDiagram object itself would play this role, but your idea definitely has some merits. I have a few questions about it though.

What about accessing the succession diagram control methods? Do you think it will be confusing that those have to be accessed via a member succession diagram? I guess we could make a wrapper around those calls.

What data would it store, other than a SuccessionDiagram object and an AttractorRepertoire object? Similarly, what functions do you think it would need?

Are there any cases in which SuccessionDiagram functions will need access to the AttractorRepertoire or vice versa? If so, it might make sense to make one a member of the other, so that it is impossible for a user to pass the wrong object. A work around might be to only access those sorts of functions through wrappers. In other languages (e.g., Java), we could enforce this kind of thing with privacy levels, but Python won't let us; we can only ask nicely with underscores, but that's probably good enough.

On the other hand, if there's ever a reason we might want to think about the attractors of one system using the succession diagram of a different system (or vice versa), having them decoupled is beneficial. Any reason you'd ever want to do this? I can't think of any.

[jgtz]

What about accessing the succession diagram control methods? ... I guess we could make a wrapper around those calls.

Yes, I was thinking on a wrapper around it.

What data would it store, other than a SuccessionDiagram object and an AttractorRepertoire object? Similarly, what functions do you think it would need?

In addition to those, I think the only required ones that make sense are the wiring diagram and the functions. We can always add thing to it, which are not important now. E.g., it could have a default initial condition, STG, the updating scheme used, network parameters (e.g. if it is actually a threshold network, the weights and thresholds).

Are there any cases in which SuccessionDiagram functions will need access to the AttractorRepertoire or vice versa? If so, it might make sense to make one a member of the other, so that it is impossible for a user to pass the wrong object.

I think we can still use the SuccessionDiagram, AttractorRepertoire dependency you proposed. On top of this, accessing them through wrappers like you suggest should be good enough in my opinion.

On the other hand, if there's ever a reason we might want to think about the attractors of one system using the succession diagram of a different system (or vice versa), having them decoupled is beneficial. Any reason you'd ever want to do this? I can't think of any.

I think its safe to assume there is not any normal circumstance you would look at this.

[jcrozum]

Ok, I think this plan makes sense. I can make the attractor repertoire and attractor space / stable module objects assuming that they will eventually be integrated into a "master" object that contains the succession diagram + attractors.

[deriteidavid]

Until this is done I'm adding an attractors_dict function/variable to the Succession class (for now in the issue38 branch) so I can work on the exports/test. Once you're done with the attractor class we can update the operations to work with that.

[jcrozum]

This has been resolved for a long time. The user now should primarily interact with the AttractorRepertoire class instead of the SuccessionDiagram class.