microsoft / Lumia-imaging-sdk

Lumia Imaging SDK is a comprehensive set of GPU/CPU imaging tools and effects that run on both mobile and desktop, with high speed and memory efficiency. Samples and extras code are included.
MIT License
75 stars 37 forks source link

How to improve blend effect quality? #14

Open Monsok opened 8 years ago

Monsok commented 8 years ago

I am developing an app that put text into a background image. The textblocks are placed on a canvas control as children and I render the canvas to PNG. Then I use the BlendEffect to blend it with a background image.

Here's my code

private async void saveChainedEffects(StorageFile file)
{
    var displayInformation = DisplayInformation.GetForCurrentView();
    var renderTargetBitmap = new RenderTargetBitmap();
    await renderTargetBitmap.RenderAsync(Textify.CanvasControl);
    var width = renderTargetBitmap.PixelWidth;
    var height = renderTargetBitmap.PixelHeight;
    IBuffer textBuffer = await renderTargetBitmap.GetPixelsAsync();
    byte[] pixels = textBuffer.ToArray();

    using (InMemoryRandomAccessStream memoryRas = new InMemoryRandomAccessStream())
    {
        //Encode foregroundtext to PNG
        var encoder = await BitmapEncoder.CreateAsync(BitmapEncoder.PngEncoderId, memoryRas);

        encoder.SetPixelData(BitmapPixelFormat.Bgra8,
                             BitmapAlphaMode.Straight,
                             (uint)width,
                             (uint)height,
                             displayInformation.LogicalDpi,
                             displayInformation.LogicalDpi,
                             pixels);

        await encoder.FlushAsync();

        IImageProvider effectBackground;

        if (SelectedEffect.Name == "No Effect")
        {
            effectBackground = imageProcessorRenderer.M_Source;
        }
        else
        {
            effectBackground = (SelectedEffect.GetEffectAsync(imageProcessorRenderer.M_Source, new Size(), new Size())).Result;
        }

        StreamImageSource streamForeground = new StreamImageSource(memoryRas.AsStream());

        //Sharpening the text has unintended consequences to I set to 0d
        using (SharpnessEffect sharpnessEffect = new SharpnessEffect(streamForeground, 0d) )
        using (BlendEffect blendEffect = new BlendEffect(effectBackground, sharpnessEffect, BlendFunction.Normal, 1.0f))
        {
            string errorMessage = null;
            Debug.WriteLine("M_SourceSize (Normalized) {0}", imageProcessorRenderer.M_SourceSize);
            Debug.WriteLine("PreviewSize {0}", imageProcessorRenderer.PreviewSize);

            try
            {
                using (var filestream = await file.OpenAsync(FileAccessMode.ReadWrite))
                using (var jpegRenderer = new JpegRenderer(blendEffect) { Size = imageProcessorRenderer.M_SourceSize, Quality = 1.0, RenderOptions = RenderOptions.Mixed })
                {
                    IBuffer jpegBuffer = await jpegRenderer.RenderAsync().AsTask().ConfigureAwait(false);
                    await filestream.WriteAsync(jpegBuffer);
                    await filestream.FlushAsync();
                }
            }
            catch (Exception exception)
            {
                errorMessage = exception.Message;
            }

            if (!string.IsNullOrEmpty(errorMessage))
            {
                var dialog = new MessageDialog(errorMessage);
                await dialog.ShowAsync();
            }
        }
    }
}

Here's the image as seen on my PC screen before it's saved to JPEG

screenshots

Then when the image is saved to JPG, there's a noticeable reduction in the quality as seen below. Enlarge the image and pay attention to the edges of the font.

textifier_20160908133852

So what are my options if I want to get as close as to the original image quality?

davidbozjak commented 7 years ago

Since you are already using the jpegRenderer.Quality parameter I don't think there is much you can do to improve the quality. Just make sure you are not leaving that to the default value which I think is at 80% at any stage during the processing, for example if you are making intermediate renderings.

Just as a test, you could try either saving a BitmapImage (to double check if it's the jpeg compression that is to blame) or try rendering on the CPU only to make sure there is no quality loss when copying the textures to the CPU for example. (JpegRenderer.RenderOptions = RenderOptions.CPUOnly)

Monsok commented 7 years ago

Thanks. I've figured out what causing the issue. It's has to do with the foreground resizing to match the background image.