KeyError in rag.learn_agglomerate(...)

[funkey]

Hi,

I am trying to run Gala (current master) on 2D images of neural tissue, but I keep getting a KeyError in learn_agglomerate(). I am using the following python script:

#!/usr/bin/python

from gala import imio, classify, features, agglo, evaluate as ev
from skimage.segmentation import relabel_sequential

print "Reading groundtruth..."
gt = imio.read_image_stack("./gt/neuron_ids0000.png")
print "Reading probability maps..."
pr = imio.read_image_stack("./prob/membrane0000.png")
print "Reading superpixels..."
sp = imio.read_image_stack("./superpixels/membrane0000.tif")

gt = relabel_sequential(gt)[0]
sp = relabel_sequential(sp)[0]

print "Creating feature managers..."
fm = features.moments.Manager()
fh = features.histogram.Manager()
fc = features.base.Composite(children=[fm, fh])

print "Creating RAG..."
rag = agglo.Rag(sp, pr, feature_manager=fc)
(X, y, w, merges) = rag.learn_agglomerate(gt, fc)[0]
y = y[:,0]
print((X.shape, y.shape))

print "Training..."
rf = classify.DefaultRandomForest().fit(X, y)

classify.save_classifier(rf, "rf.dat")

on this data: http://lego.slyip.net/gala_mwe.tar.gz. This is the traceback of the error I get:

Traceback (most recent call last):
  File "./train.py", line 23, in <module>
    (X, y, w, merges) = rag.learn_agglomerate(gt, fc)[0]
  File "/usr/local/lib/python2.7/dist-packages/gala-0.3dev-py2.7-linux-x86_64.egg/gala/agglo.py", line 1264, in learn_agglomerate
    learning_mode, labeling_mode))
  File "/usr/local/lib/python2.7/dist-packages/gala-0.3dev-py2.7-linux-x86_64.egg/gala/agglo.py", line 1411, in _learn_agglomerate
    node_id = g.merge_nodes(n1, n2, merge_priority)
  File "/usr/local/lib/python2.7/dist-packages/gala-0.3dev-py2.7-linux-x86_64.egg/gala/agglo.py", line 1582, in merge_nodes
    self.refine_post_merge_boundaries(n1, n2, sp2segment)
  File "/usr/local/lib/python2.7/dist-packages/gala-0.3dev-py2.7-linux-x86_64.egg/gala/agglo.py", line 1636, in refine_post_merge_boundaries
    self.update_merge_queue(u, v)
  File "/usr/local/lib/python2.7/dist-packages/gala-0.3dev-py2.7-linux-x86_64.egg/gala/agglo.py", line 1742, in update_merge_queue
    w = self.merge_priority_function(self,u,v)
  File "/usr/local/lib/python2.7/dist-packages/gala-0.3dev-py2.7-linux-x86_64.egg/gala/agglo.py", line 207, in predict
    features = feature_extractor(g, n1, n2)
  File "/usr/local/lib/python2.7/dist-packages/gala-0.3dev-py2.7-linux-x86_64.egg/gala/features/base.py", line 9, in __call__
    return self.compute_features(g, n1, n2)
  File "/usr/local/lib/python2.7/dist-packages/gala-0.3dev-py2.7-linux-x86_64.egg/gala/features/base.py", line 19, in compute_features
    if g.node[n1]['size'] > g.node[n2]['size']:
KeyError: 'size'
Exception TypeError: TypeError("'NoneType' object is not callable",) in <bound method UmfpackContext.new_del of <scipy.sparse.linalg.dsolve.umfpack.umfpack.UmfpackContext object at 0x7f1a02408610>> ignored

Any help would be greatly appreciated!

Cheers, Jan

[jni]

Hey @funkey! Great to see you're trying out gala! =)

The label "0" in the superpixels has a special meaning in gala. It's supposed to be the boundary between superpixels. This is for historical reasons, because we were originally generating our watersheds with Matlab, which produces a 0 boundary on the watershed lines. There's actually a bug that I can't track down with 0-boundaries, so I'm actually just in the process of deprecating this (see this recent post on the mailing list).

In your case, your superpixel labels start at 0, which is even worse than having zero boundaries. =P The simple fix is to add the line sp += 1 before or after relabel_sequential. =) You should probably do the same for the ground truth.

Sorry for the confusion! This magic behaviour will go away very soon.

Let me know if that resolves your issue!

[funkey]

Hi Juan!

Thank you very much for you prompt reply -- that solved the problem indeed. The only remaining error is

Exception TypeError: TypeError("'NoneType' object is not callable",) in <bound method UmfpackContext.new_del of <scipy.sparse.linalg.dsolve.umfpack.umfpack.UmfpackContext object at 0x7f2503a935d0>> ignored

but the training and merging seems to work anyway.

[jni]

@funkey That error comes from an interaction between certain versions of scikit-image and scipy. It should be resolved if you update scipy. But it doesn't affect anything, it is merely an import messing with a class destructor, followed by an error when the destructor is called at the end of execution.