need vertical scales and value labels

[cannam]

Extracting, say, a pitch track is a very unrevealing exercise -- we deliberately have no way to overlay it onto a spectrogram (as one might do in SV and is done automatically in Tony) so we really need some sort of vertical scale, and/or to have the most salient/common pitch values labelled where they appear.

[cannam]

We now have a vertical scale on the left and a highlight line to identify the value beneath the playback pointer. Both of these appear on line/vector-type layers only. Most of the work is in waves-ui.

The vertical scale is OK I think, but having spent some time thinking again about how to work out which values to label, I want to go back and update the code that does this in SV and introduce some more extensive tests for it. I think I may do that this morning.

The highlight/crosshair function I also quite like, but we probably do need to be able to toggle it. Perhaps the toggle should also toggle visibility of the scale.

Next priority here will be to get both of them working for matrix shapes. They have a different notion of vertical scale - it's kind of implicit in the labels for the feature bins and is not explicitly linear (or logarithmic or anything else).

[LucasThompson]

Great stuff! There's a slight quirk with the cursor playing over the vertical scale - perhaps the width of the vertical scale should be subtracted from the original width and everything shifted right by that distance. Not much of an issue though really.

Just tried it on my phone and it is easily legible - combined with the highlight layer this is a huge step forward in functionality. In terms of toggling, I think it would make sense if the crosshair and the scale were paired.

[cannam]

If you shifted everything right, wouldn't that mean it was no longer aligned with the waveform track?

The scale has an opaque white rectangular background, to block out the shapes beneath. Stacking order in SVG is determined (I think) implicitly, purely on the order in which the elements appear in the SVG document. We could prevent the cursor appearing on the scale by putting its layer earlier in the document than the scale layer. I think.

[LucasThompson]

Ha, yeah good point. Shifting doesn't make sense then. Stacking the shapes / layers differently as you say would probably be fine, and perhaps even an easier "fix". I'll take a look.

[cannam]

You know how you take something that is basically working, and you think "I'll just tidy this up", and next thing you know you're surrounded by opened cans of worms? Metaphorically of course.

When I said "I want to go back and update the code that [calculates which values to label and decides how to label them] in SV and introduce some more extensive tests for it. I think I may do that this morning", what I really meant was "this month".

Not really -- hopefully get it done today or tomorrow morning. Definitely more complicated than it was though.

[cannam]

I'm finally happy (or happy enough for now) with how this code works in SV -- just need to backport it into the Javascript again!

[cannam]

We now have a vertical scale and value highlight that I'm happy with for now, but only for line/vector layers. Still open for matrices.

[cannam]

Just added a vertical scale for matrices (in the "more-vertical-scales" branch of my own fork) but so far only using the bin count to provide the scale. Here we run slapbang into the next grave failing of our feature shaping code: it doesn't preserve the bin labels for matrix features, which are the only clues we have for what the vertical scale should look like.

[cannam]

There are also some other aspects of the output descriptor that we should be taking into account in deciding how to draw things.

For example, in a curve shape we are currently establishing the min/max values for the scale and vertical axis range by going through the feature data and literally finding the minimum and maximum. But if the output descriptor "hasKnownExtents", we should probably be using those extents instead as they are possibly going to be more sensible for display. (For matrices, the output descriptor extents map to the colour scale rather than the Y axis scale.)

[cannam]

So, how to get the bin values brought through?

In the piper-js code, the natural thing to do appears to be to add a descriptor property to the object returned by reshape() and collect() in HigherLevelUtilities. This object currently has shape and collected properties, where collected holds the reshaped features.

However, that won't work very nicely in Ugly Duckling's FeatureUtilities, where the reshape logic (toKnownShape) does not return an equivalent object to that returned by collect but instead returns only the collected part of it.

To me, returning something like this from collect seems kind of OK

{
  shape: "matrix",
  descriptor,
  collected: {
    startTime,
    stepDuration,
    data
  }
}

and returning something like this seems kind of maybe sort of just about OK

{
  shape: "matrix",
  collected: {
    startTime,
    stepDuration,
    binNames,
    extents,
    unit,
    data
  }
}

but this would seem kind of wrong, like mixing your data and metadata

{
  shape: "matrix",
  collected: {
    startTime,
    stepDuration,
    descriptor,
    data
  }
}

[cannam]

Lucas has just pointed out that collect in HigherLevelUtilities is not actually used.

What is used is the PiperSimpleClient which implements SimpleService which has a collect method that returns a Promise<SimpleResponse>, and SimpleResponse already includes an outputDescriptor field.

Then, in Ugly Duckling, toKnownShape takes this SimpleResponse as its argument. At the moment, for most shapes it just returns the same feature structure as found in piper-js, which doesn't contain the output descriptor. But it could return enhanced or augmented structures, and in fact it does do so already for note shapes which are not handled in a dedicated way in collect but do want to be handled separately by Ugly Duckling.

[cannam]

Have now followed this through, with a vertical binned scale for the matrix shape in Ugly Duckling.

Required adding a new binned ("discrete") scale in Waves UI and corresponding new providers in Ugly Duckling.

Well, maybe it didn't "require" that but that's what I did.

[cannam]

I think this is good enough to close now.