Need help DrawInk(inkStrokes) and StrokeContainer.BoundRect resizing issue

[Ponant]

Hi, I would really appreciate some guidance or solution as I am struggling on a seemingly simple issue about saving a set of strokes from a UWP InkCanvas to a PNG file, with the condition that I want the PNG image to fit the strokes boundaries and not the whole InkCanvas. For some reason, rendering to PNG works well for some collection of strokes, but at other times it cuts the image at the boundaries, either left or right, or top or bottom and I am unable to find a solution. I provide here the necessary code to make it run as well as a sample ink file which reproduces the issue. The red rectangle in the code is here to show the BoundRect of the StrokeContainer. Here is one Gif with embedded ISF: .

You load it and save as PNG and check out the borders. Thanks a lot if you can help me.

XAML

<Grid Background="{ThemeResource ApplicationPageBackgroundThemeBrush}">
        <Grid.RowDefinitions>
            <RowDefinition Height="Auto" />
            <RowDefinition Height="*" />
        </Grid.RowDefinitions>

        <Grid x:Name="topGrid" Grid.Row="1">
            <Grid x:Name="inkCanvasGrid" BorderBrush="Yellow" BorderThickness="2"
                  Background="White">
                <InkCanvas x:Name="inkCanvas"/>
                <Rectangle x:Name="rectangle" Fill="Red" Opacity="0.5" Width="100" Height="100" HorizontalAlignment="Left" VerticalAlignment="Top"/>
            </Grid>
        </Grid>

        <StackPanel>
            <Button Content="Load" Click="LoadButton_Click"/>
            <Button Content="Save" Click="SaveButton_Click"/>
        </StackPanel>
    </Grid>

C#

private async void SaveButton_Click(object sender, RoutedEventArgs e)
        {
            if (inkCanvas.InkPresenter.StrokeContainer.GetStrokes().Any())
            {
                var fileSave = new FileSavePicker();

                fileSave.FileTypeChoices.Add("PNG", new string[] { ".png" });

                fileSave.SuggestedStartLocation = PickerLocationId.PicturesLibrary;

                var storageFile = await fileSave.PickSaveFileAsync();

                if (storageFile != null)
                {
                    //dataText.Text = $"selected {storageFile.FileType}";
                    using (var stream = await storageFile.OpenAsync(FileAccessMode.ReadWrite))
                    {
                        await SaveInkToStream(stream, storageFile.FileType);
                    }
                }
            }
        }

        private async Task SaveInkToStream(IRandomAccessStream stream, string fileType)
        {
            //var strokeContainerWidth = inkCanvas.ActualWidth;
            //var strokeContainerHeight = inkCanvas.ActualHeight;

            var strokeContainerBoundRect = inkCanvas.InkPresenter.StrokeContainer.BoundingRect;

            rectangle.RenderTransform = new TranslateTransform
            {
                X = strokeContainerBoundRect.X,
                Y = strokeContainerBoundRect.Y,

            };
            rectangle.Height = strokeContainerBoundRect.Height;
            rectangle.Width = strokeContainerBoundRect.Width;

            CanvasDevice device = CanvasDevice.GetSharedDevice();

            using (var renderTarget = new CanvasRenderTarget(device, (float)strokeContainerBoundRect.Width, (float)strokeContainerBoundRect.Height, 96f))
            {

                using (CanvasDrawingSession ds = renderTarget.CreateDrawingSession())
                {
                    ds.Transform = new Matrix3x2() { M11 = 1, M22 = 1, Translation = new Vector2((float)-strokeContainerBoundRect.X, (float)-strokeContainerBoundRect.Y)};
                    ds.Clear(Colors.White);
                    ds.DrawInk(inkCanvas.InkPresenter.StrokeContainer.GetStrokes());
                }

                await renderTarget.SaveAsync(stream, CanvasBitmapFileFormat.Png, 1f);
            }
        }

        private async void LoadButton_Click(object sender, RoutedEventArgs e)
        {
            FileOpenPicker openPicker = new FileOpenPicker();
            openPicker.FileTypeFilter.Clear();
            openPicker.FileTypeFilter.Add(".isf");
            openPicker.FileTypeFilter.Add(".gif");

            openPicker.SuggestedStartLocation = PickerLocationId.PicturesLibrary;

            StorageFile file = await openPicker.PickSingleFileAsync();

            if (file != null)
            {
                IRandomAccessStream stream = await file.OpenAsync(FileAccessMode.Read);

                // Read from file.
                using (var inputStream = stream.GetInputStreamAt(0))
                {
                    await inkCanvas.InkPresenter.StrokeContainer.LoadAsync(inputStream);
                }
                stream.Dispose();

            }
        }

[clandrew]

The PNG file will be sized according to strokeContainerBoundRect, so I wonder why strokeContainerBoundRect would be under-size, or if there is something else going on.

Do you know where we can find the sample ink file to reproduce the issue? Sorry, can't seem to find it in the bug.

[Ponant]

@clandrew , did you get the file? It is the first image you see (a gif). Right click and save...

[Ponant]

[Ponant]

@clandrew , I think this is an issue of the StrokeContainer rather than Win2D. You may have noticed that already the gif provided is truncated on the top and bottom, while if you open them with the uwp app you will see the drawing fully embedded throughout the boundaries.. The draw itself is made with the same uwp. I suspect, thus, that the Gif rendering (Gif+ISF embedded) also uses the StorkeContainer.BoundRect. I do not know where to report this as I think UWP is not open source and hence not on the public Github. However, in any case I would appreciate a solution and possibly a workaround (in code if possible), so that I can proceed in the meanwhile. Cheers.

[shawnhar]

Sorry, I don't understand what you are saying here. What does "while if you open them with the uwp app" mean? I don't know what "them" refers to, or which UWP app you are using.

You provided a png file that has been clipped, but the code you are having trouble with operates on ink strokes to generate that png - it does not use the png as input! To repro this we need to know how to run your code, which means sharing a complete program + all the necessary input data for it. Just seeing the output from after a problem has already occurred is not enough.

May I suggest that you add some code in your app to draw the bounding rect over the top of the ink? That should easily narrow down whether the problem is in your saving code or if you have a wrong bounding rect in the first place.

[Ponant]

@shawnhar , the file provided is a gif file, that I saved from an InkCanvas from another app (more complex but I guarantee you this issue does not come from my other app).

The file I provided is not a PNG and it is not clipped by me, but rather it is clipped by having saved the file to gif. By them, I meant the strokes, sorry for the confusion. The rectangle is automatically drawn with the code above. Basically, to reproduce the issue: 1) Right click the image (gif!) and save it as is. 2) Create a UWP app (Anniversary build in my case) in VS (2017 community) 3) Copy paste the XAML above in MainPage.Xaml 4) Copy paste the c# code above in MainPage.cs 5) Run the app 6) You will see a button to load a gif file. Load the gif file provided here 7) Click Save, and it will save it to PNG 8) Observe the clipping

[Ponant]

@shawnhar , to avoid the confusion, the gif file above is clipped as you can see, but if you load it into an inkcanvas it will appear in full. And saving it to PNG will clip it again. I think the steps to reproduce are clear now.

[shawnhar]

Sorry, I don't have time to put together a test app from scratch by piecing together bits of instructions. Please provide a complete app including all necessary data that I can compile and run to repro your problem.

[Ponant]

That is 2-5 minutes max @shawnhar and you do not have the time? I decided not to make a repo on GitHub because the steps to reproduce are obvious and short.

[shawnhar]

I would be happy to help with your problem if you can provide a good repro for me to look at. Feel free to reactivate this issue if you are able to do that.

[Ponant]

@shawnhar , I am not sure if you work for Microsoft because I find this behaviour unprofessional, because in the end you refuse to address the issue and even close it, hence closing your eyes on a possible bug. Furthermore, if you think about it, the time you clone a repo is virtually identical to the time taken for 2 copy paste and one right click to save a gif file. Anyway you are not obliged to help.

[shawnhar]

Yes, I work for Microsoft (if in doubt, you can check that in the github profile for all Microsoft employees who are using github in a professional capacity).

I'm sorry that you are not happy with the outcome of this issue. Like I said, the Win2D team will be happy to help debug your problem if you are able to provide a repro app for it.

[Ponant]

I am not sorry about the outcome, but the attitude because it is common not provide a repo every time if the steps to reproduce are short. And I am sorry, but the time excuse is not a clever one. Here is the repo, for the sake of clarifying this. https://github.com/Ponant/InkDrawvsStorkeContainer

[shawnhar]

Thanks for the repro.

The problem appears to be that InkStroke.GetBounds is not accounting for bezier curvature, so it only reports bounds for the main control points of the strokes. I will ask the ink team whether this is a bug or the expected behavior.

A workaround would be to convert the ink strokes into Win2D geometry (using CanvasGeometry.CreateInk) and then use Win2D geometry APIs to query their bounds. That will be accurate for all pen ink strokes, but will not work if you are using pencil ink style.

[Ponant]

I think the problem might also be that it does not account for pressure and rotation of the pentip. I expect it to be a bug because even the gif file is clipped when viewed with Windows's photo viewer. I will try your workaround and I am also creating my own measure of the bounds to supress this defect. I look forward for the feedback from the ink team.

[shawnhar]

Ink team confirmed that this is a known issue. I don't think it is affected by pressure or rotation, as your example has pressure disabled. Padding the bounds is one option, but using Win2D geometry will give a 100% accurate measure.

[Ponant]

I tried your suggestion and it works well with ComputeBounds. You are right about the pressure and rotation; I made this draw for testing some time ago. Is it possible to know what is the underlying issue they are facing? I ask this because I am also rendering the ink strokes to SVG and a similar problem occurs but here I cannot use Win2D (I think). In the meantime I am computing the bounds on my own for the export to SVG and it works great but it requires me to loop through all strokes and inkpoints in those strokes and get the max and min to get the bound values.