How to reproduce the HED-owt-ucm results for voc2012

[Dingry]

Hi, I download the HED and gPb-owt-ucm code for reproducing the voc2012 segmentation segments. But I found that the gPb produce the 8-orientation boundary probabilities, which is needed in owt, while HED not. So I wonder how can I input the HED result into owt algorithms. Thanks!

[jyhjinghwang]

Hi Dingry,

Since there hasn't been a single codebase that does the job, I used two toolboxes: 1) To generate oriented contours. Piotr's Matlab Toolbox: https://github.com/pdollar/toolbox 2) To generate owt-ucm. MCG: https://github.com/jponttuset/mcg Here's a snippet of code I put together (in Matlab) to generate HED-owt-ucm from HED.

  % Add Piotr's toolbox path.
  addpath(genpath('./toolbox/'));

  % Add MCG path.
  addpath(genpath('./mcg-2.0/full/src/ucms/'));
  addpath(genpath('./mcg-2.0/full/lib/'));

  % Load HED contours as variable E.
  ... 

  % Calculate orientations of contours.
  [Ox,Oy] = gradient2(convTri(E,4));
  [Oxx,~] = gradient2(Ox);
  [Oxy,Oyy] = gradient2(Oy);
  O = mod(atan(Oyy.*sign(-Oxy)./(Oxx+1e-5)),pi);

  % Perform oriented watershed transform.
  [owt2, superpixels] = contours2OWT(E, O);

  % Perform spectral globalization.
  sPb = spectralPb_fast(owt2);
  sPb = (sPb - min(sPb(:))) / (max(sPb(:)) - min(sPb(:)));

  % Calculate ucm2.
  ucm2 = double(ucm_mean_pb(owt2 + sPb, superpixels));

  % Calculate the final segmentation given threshold th.
  labels2 = bwlabel(ucm2 <= th);
  labels = labels2(2:2:end, 2:2:end);

Hope this helps.

Best, Jyh-Jing

[Dingry]

Thanks! It works!

[ifeherva]

"% Load HED contours as variable E."

I am generating the HED contours with my own pytorch model. What format is needed for your algorithm to work? E.g. HxW uint8? Or WxH float?

[jyhjinghwang]

Hi @ifeherva ,

It's the same as the input images, which are HxW uint8.

Jyh-Jing

[ifeherva]

Thanks a lot, I got it to work as well. (Had to convert E to single otherwise conv did not work).

What was the threshold you used (th)? I vaguely remember 0.3, but I might be wrong.

[jyhjinghwang]

Glad to know it worked for you! (Yes, conversion to single would be required for some platforms.) We used 0.95 for the ucm threshold.

[ifeherva]

I still have troubles reproducing the cluster labels. I am using matlab2020a and I do not really understand what is going on, but I definitely dont get meaningful results using the code above.

In fact 'ucm2' has a bunch of zeros where the clusters should be and the 'edges' are superhigh values like 53 or 191 so the check against .9 does not make any difference. Thus bwlabel creates hundreds of clusters. I think the thresholding is not working properly so I get too many edges, do you have any idea where the bug might be? maybe the values in ucm2 are not normalized (in my matlab version)? Thanks a lot!

[ifeherva]

Update: owt2 needs to be normalized before calling ucm_mean_pb(). Fixing that seems to be producing the same output as published.

[ashwinipokle]

Hi @jyhjinghwang,

I am using another contour detector instead of HED. I am loading these images as H X W float (uint8 gave me an error in convTri(E,4))

However, upon running this script with various threshold values (0.5, 0.4, 0.2, 0.1, 0.05), I always end up with very few labels (mostly 1 with very few 2 and 3).

Here are few sample images I am loading in E. Could you tell me what could be going wrong?

I tried normalizing owt2 as @ifeherva suggested but that doesn't help too.

[jyhjinghwang]

Hi @ashwinipokle,

It's hard to know what went wrong by looking at these two images--they seem totally fine. I'd suggest you print out the histogram of E, normalized owt2, and normalized sPb. I guess lots of contour intensities are cluttered at a very low range?

Regards, Jyh-Jing