DrawInk Fails to DrawAsHighlighter

[zlstringham]

Considering the following CanvasControl.Draw event handler:

void _CanvasControl_OnDraw(CanvasControl sender, CanvasDrawEventArgs args)
{
    var inkStrokeBuilder = new InkStrokeBuilder();
    inkStrokeBuilder.SetDefaultDrawingAttributes(_inkDrawingAttributes);
    var stroke = inkStrokeBuilder.CreateStroke(PointCollections);
    var strokes = new List<InkStroke> { stroke };

    args.DrawingSession.DrawInk(strokes);
}

This renders fine if _inkDrawingAttributes.DrawAsHighlighter is false.\ However, if it is true, nothing is displayed.

[shawnhar]

You can't use highlighter ink on a partially transparent CanvasControl layer which is then composed over the top of some other image. You must draw the highlighter directly onto a surface which already contains whatever you are trying to highlight, which means that underlying image must also be drawn to the same CanvasControl a the ink.

Previous discussion on this:

https://github.com/Microsoft/Win2D/issues/462 https://github.com/Microsoft/Win2D/issues/428 https://stackoverflow.com/questions/36397638/win2d-uwp-wont-save-a-highlight-stroke-in-an-inkcanvas

[zlstringham]

@shawnhar However, I could create a transparent InkCanvas control and do inkCanvas.InkPresenter.StrokeContainer.AddStroke(stroke), where stroke is DrawAsHighlighter = true.

I don't understand the fundamental difference in the operations, however the InkCanvas method is not feasible for me due to InkCanvas limitations; i.e., I can't create 100+ InkCanvas instances, but I can create 100+ CanvasControl instances.

Working with an extensive codebase which already has an implementation for rendering these markup, I specifically pulled in Win2D to render ink markup. It currently uses the InkCanvas method for pen and highlighter markups, but crashes once a certain threshold is crossed. I was successful in applying CanvasControl in place of InkCanvas for pen markup, and I would prefer, if possible, to not have to refactor everything to render the whole scene in Win2D simply for highlighter markup.

[shawnhar]

This article explains some of the background of how alpha blending and image composition work: https://blogs.msdn.microsoft.com/shawnhar/2009/11/07/premultiplied-alpha-and-image-composition/

Drawing non-highlighter ink to a CanvasControl, and then getting XAML to compose that CanvasControl over the top of some other image, works correctly because both non-highlighter ink and XAML composition use premultiplied alpha blending (which is also commonly referred to as source-over blending):

• first the ink blends to a CanvasControl that was cleared to transparent: srcover(0, ink)
• and then XAML blends the CanvasControl over the background image: srcover(image, canvas)
• and because srcover blending is associative, the result is srcover(image, srcover(0, ink))
• which is the same thing as srcover(image, ink)

Highlighter ink does not work this way because it uses a min blend function:

• ink blends to a transparent CanvasControl: min(0, ink)
• then XAML blends the CanvasControl over the background image: srcover(image, canvas)
• result is srcover(image, min(0, ink))
• but min(0, anything) is always 0
• so the result is srcover(image, 0)
• which does nothing

In short, you cannot do this. Either don't use CanvasControl, or draw your background image into the CanvasControl, then ink over the top, rather than relying on XAML to compose these things for you.

[christianrr]

Found a possible solution to this problem:

1) Create a separate canvas control for the highlighter strokes and one for the normal "pen" strokes (the important thing is to set the CompositionMode of the highlighter's texture to MinBlend to blend the highlighter strokes with underlying background):

<xaml:CanvasControl x:Name="UxHighlighterInkTexture" Draw="OnRepaintHighlighterInkTexture" CompositeMode="MinBlend" /> <xaml:CanvasControl x:Name="UxInkTexture" Draw="OnRepaintInkTexture" SizeChanged="OnInkTextureSizeChanged"/>

2) In the second stage, draw the highlighter strokes on the highlighter canvas control in a two-step approach: First, draw the strokes as non-highlighter strokes into the texure to form the background to make highlighter strokes appear and then in a second stage draw the highlighter strokes on top of the non-highlighter strokes to blend all the strokes into each other.

// render highlighter strokes
var highlighterStrokes = strokesToRender.Where(stroke => stroke.DrawingAttributes.DrawAsHighlighter).ToList();

// set DrawAsHighlighter to false temporary
foreach (var stroke in highlighterStrokes)
{
    var attributes = stroke.DrawingAttributes.Clone();
    attributes.DrawAsHighlighter = false;
    stroke.DrawingAttributes = attributes;
}

// draw highlighter strokes (1st pass) - background to make texture blending work
session.DrawInk(highlighterStrokes);

// restore attributes
foreach (var stroke in highlighterStrokes)
{
    var attributes = stroke.DrawingAttributes.Clone();
    attributes.DrawAsHighlighter = true;
    stroke.DrawingAttributes = attributes;
}

// draw highlighter strokes (2nd pass) - adds blending of strokes
session.DrawInk(highlighterStrokes);

With that solution, I was able to reproduce the normal InkCanvas behavior. The first step blends your highlighter strokes with any underlying content, while the two-step approach in the second stage blends the internal highlighter strokes against each other.