Namespace: multi-level segmentation

[bmcfee]

cc @urinieto

We should define a namespace for multi-level segmentations.

I'm not sure how exactly this should look yet, so please chime in. A few things to keep in mind:

• should a boundary be declared once with a corresponding depth, or should we redundantly encode a boundary at all depths to which it belongs?
  • I think the latter, since it becomes easier to query/filter. Also it doesn't require that different layers be hierarchical. This could cause headaches if we later link this into https://github.com/urinieto/hier_eval though.
• how should data be packed? overload the confidence field to encode depth, or pack the value field as a structure {label: 'segment label', level: 'segment level'}?
  • I see good arguments for both. Overloading confidence is a little unnatural in jams speak, but it does make things easy to filter down to standard (flat) evaluation methods. OTOH, the struct coding is more explicit and intuitive.

[urinieto]

should a boundary be declared once with a corresponding depth, or should we redundantly encode a boundary at all depths to which it belongs?

I would also vote for the latter, since it feels more natural and I don't think the extra bytes would significantly impact the size/performance.

how should data be packed? overload the confidence field to encode depth, or pack the value field as a structure {label: 'segment label', level: 'segment level'}?

I would vote for the latter, since overloading the confidence field seems a hack that might not be easy to understand (I have experience on this with msaf).

Overall, this is an interesting discussion, since it would be great to also add hierarchical evaluation in mir_eval connected to hierarchical JAMS.

[bmcfee]

Agreed on both counts.

As for for hierarchy evaluation: yup! That's where I'm going with this.

We'll need to resolve the ordering of observations by level though. For example, the SALAMI annotations would have levels function, large, and small, and in this case, sorting alphabetically happens to work out. I'm not convinced that that holds in general though. Should we add a third depth field that explicitly encodes ordering?

OTOH, maybe we can just explicitly assert that levels are ordered according to python sort semantics applied to level and not add a third field. It's then up to the user to ensure that sorting by level does the right thing.

[urinieto]

As for for hierarchy evaluation: yup! That's where I'm going with this.

ALRIGHT :)

Could we have implicit order in the levels based on the number of segments? This way we wouldn't need an extra variable, nor do hacky things based on level names.

[urinieto]

Also, I'm not sure if you remember my mir_eval fork with hierarchical evaluations. Some parts might be reusable.

[bmcfee]

Sure; although I think that bit can be rewritten in a simpler form without so much objectiness. We should settle that first (in a separate thread) before finalizing the hierarchy schema.