`evaluate_cut` running slow ?

[AjayTalati]

Hi @wmayner,

can you give some advice/help why I find it takes much longer to run compute.evaluate_cut compared to simply running compute.big_mip ?

Here's a minimal script you can run, and also a net you can download

https://www.dropbox.com/s/705c2ne4gpj13qo/network_4_nodes_dev_pyphi.json?dl=0

to reproduce it exactly, (but I get the same behaviour for all other subsystems too).

import pyphi
from pyphi.compute.big_phi import evaluate_cut
import json
from random import randint
from time import time

##---------------------------------------------------
# Import net and setup system 
##---------------------------------------------------

import json

path_to_saved_net = "/home/ajay/Downloads/network.json"
with open(path_to_saved_net) as data_file:    
    data = json.load(data_file)

tpm = data['tpm']
cm = data['cm']
state = data['state']

network = pyphi.Network(tpm, connectivity_matrix=cm)
subsystem = pyphi.Subsystem(network, state, range(network.size)) 
#subsystem.node_indices

pyphi.config.CACHE_BIGMIPS = False
pyphi.config.PARALLEL_CUT_EVALUATION = False

##---------------------------------------------------
# Simple Big mip calculation
##---------------------------------------------------

def compute_big_mip(subsystem):
    start = time()
    for i in range (0,100):
        mip = pyphi.compute.big_mip( subsystem ) 
    print(time()-start)

##---------------------------------------------------
# Simple Big mip calculation and a random cut
##---------------------------------------------------

unpartitioned_constellation = pyphi.compute.constellation(subsystem)

all_bipartitions = pyphi.utils.directed_bipartition( subsystem.node_indices )[1:-1] # can be precomputed when the environment is initialized
num_all_directed_bipartitions = len(all_bipartitions) 

def compute_big_mip_and_a_random_cut( subsystem ):
    start = time()
    for t in range (0,100): 
        mip = pyphi.compute.big_mip(subsystem) 
        i = randint(0, num_all_directed_bipartitions-1)
        rand_bipartition = all_bipartitions[ i : i+1 ] # random directed bipartition
        random_cut = [ pyphi.models.cuts.Cut ( bipartition[0], bipartition[1] ) for bipartition in rand_bipartition ] # random_cut
        mip_of_rand_cut = evaluate_cut(subsystem, random_cut[0], unpartitioned_constellation) # pyphi/compute/big_phi.py#L27           
    print(time()-start)

##---------------------------------------------------
# Compare timings 
##---------------------------------------------------

>>> compute_big_mip(subsystem)
10.356880187988281
>>> 
>>> compute_big_mip_and_a_random_cut( subsystem )
39.57922887802124

So my question is why does it take 3 times as long to evaluate a single random cut, compare to evaluating the big_mip for a full system which requires searching over many cuts?

If I can sort this out I'll be able to run some experiments, and can show you how to apply ML here.

Thanks a lot for your help,

Aj

[AjayTalati]

Update, I think this is due to the way pyphi catches things, (but I don't know how to turn this on and off yet)?

I generated and stored 100, random tpms and states, and then ran the above on them, and got much more common sense times,

>>> compute_big_mip()
33.894142627716064
>>> compute_big_mip_and_a_random_cut()
31.76900362968445

So I'll close this, as it seems like it's really use case confusion, rather than problem with pyphi

[AjayTalati]

Oh dear maybe I spoke too soon? Now if I calculate the random cut only, it still seems very slow,

>>> def compute_a_random_cut_only():
...     start = time()
...     for t in range (0,100): 
...         network = pyphi.Network(tpms[t], connectivity_matrix=cms[t])
...         subsystem = pyphi.Subsystem(network, states[t], range(network.size)) 
...         #mip = pyphi.compute.big_mip(subsystem) 
...         i = randint(0, num_all_directed_bipartitions-1)
...         rand_bipartition = all_bipartitions[ i : i+1 ] # random directed bipartition
...         random_cut = [ pyphi.models.cuts.Cut ( bipartition[0], bipartition[1] ) for bipartition in rand_bipartition ] # random_cut
...         unpartitioned_constellation = pyphi.compute.constellation(subsystem)
...         mip_of_rand_cut = evaluate_cut(subsystem, random_cut[0], unpartitioned_constellation) # pyphi/compute/big_phi.py#L27           
...     print(time()-start)
... 
>>> compute_a_random_cut_only()
30.439067840576172

I would have expected it to be much faster if it's only got to calculate one random cut, and not the full subsystem's big_mip. Any ideas where all the computation time is going?

[wmayner]

Hi Ajay,

I finally had a chance to look into this, and I can't seem to reproduce it. I wrote a similar script, and I see results more in line with what we'd expect:

Timing functions (4 nodes, 100 iterations)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
big_mip: done.
  Mean time: 0.6552s
naive_big_mip: done.
  Mean time: 0.7885s
unpartitioned_constellation: done.
  Mean time: 0.0965s
random_cut: done.
  Mean time: 0.0680s

When the script is run with 5 nodes rather than 4, I get similar results:

Timing functions (4 nodes, 100 iterations)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
big_mip: done.
  Mean time: 31.5975s
naive_big_mip: done.
  Mean time: 32.6524s
unpartitioned_constellation: done.
  Mean time: 2.0442s
random_cut: done.
  Mean time: 0.9371s

[AjayTalati]

Hi Will, @wmayner,

sorry for the late reply - I'm not ignoring you - I got caught up in something else. I'm looking into this now, and going to reply to your email soon.

It's great that you've got the pyphi-users forum rolling - maybe you want to put a link to it on the front page here to let new users know it exists?

Best,

Aj