Add Tests for System.Drawing.Common

[mellinoe]

This issue tracks porting some set of tests from mono's test suite, covering the portions of System.Drawing that we support on .NET Core.

Mono's test cases are in this folder: https://github.com/mono/mono/tree/master/mcs/class/System.Drawing/Test

We most likely want to convert the most useful tests from all of the sections here, with the exception of System.Drawing.Design, which we aren't going to support right now on .NET Core (it is mainly related to designer / WinForms support).

Mono's tests use NUnit, so we will need to convert them to Xunit when copying them.

Additionally, I've identified that there will need to be some functional changes made to the tests themselves, as they do not pass against the .NET Framework implementation. We consider the .NET Framework implementation to be the compatibility baseline, so we should change the tests to accomodate it, rather than the other way around. The test failures seemed mainly related to very small, subtle differences in things like floating-point precision, color values, offsets, etc. We should do the following when we have both Windows and Unix implementations up and running:

• Ensure that all tests have enough "leniency" to accomodate floating-point differences (and other minor inconsistencies) where unavoidable.
• Ensure that all minor inconsistencies are acceptable or otherwise very difficult / problematic to fix.
• Ensure that all minor inconsistencies are listed somewhere, so that we can mention it both in the tests, and in the user documentation.

@hughbe @qmfrederik @marek-safar

---

Current Status

Code coverage: Note that there is a large amount of internal and debug-only code which distorts these numbers. When the coverage is generally high, we can clean out a lot of dead code and then get more accurate data.

Date	Branch Coverage	Line Coverage
6/26/2017	33%	25%
7/11/2017	49%	54%
7/17/2017	55%	60%
8/2/2017	58%	66%
8/25/2017	62%	71%
9/21/2017	64.6%	75.3%

Namespaces and coverage, as of 9/21/2017:

• System.Drawing: 76.3%
• System.Drawing.Drawing2D: 93.8%
• System.Drawing.Imaging: 86.1%
• System.Drawing.Printing: 62.8%
• System.Drawing.Text: 96%

[mellinoe]

I've written some tests for AdjustableArrowCap ( dotnet/corefx#21911 ), and will work on some tests for CustomLineCap when I have time this week.

[KostaVlev]

I will try to write tests and convert the old once for GraphicsPathIterator also. :)

[hughbe]

Awesome. I'm busy this weekend and was this week but I'm working on tests for Graphics Once yours and @norek's PRs are in we should be getting to a good level of coverage - lots more to be done though!

[KostaVlev]

I have a little trouble here need somebody to confirm the following is correct.

The test method I wrote:

     public static IEnumerable<object[]> CopyData_StartEndIndexesOutOfRange_TestData()
    {
        yield return new object[] { new PointF[3], new byte[3], -1, 2 };
        yield return new object[] { new PointF[3], new byte[3], int.MinValue, 2 };
        yield return new object[] { new PointF[3], new byte[3], 0, 3 };
        yield return new object[] { new PointF[3], new byte[3], 0, int.MaxValue };
        yield return new object[] { new PointF[3], new byte[3], 2, 0 };
    }
    [ConditionalTheory(nameof(PlatformDetection) + "." + nameof(PlatformDetection.IsNotWindowsNanoServer))]
    [MemberData(nameof(CopyData_StartEndIndexesOutOfRange_TestData))]
    public void CopyData_StartEndIndexesOutOfRange_ReturnsExpeced(PointF[] points, byte[] types, int startIndex, int endIndex)
    {
        PointF[] resultPoints = new PointF[points.Length];
        byte[] resultTypes = new byte[points.Length];

        using (GraphicsPath gp = new GraphicsPath(points, types))
        using (GraphicsPathIterator gpi = new GraphicsPathIterator(gp))
        {
            Assert.Equal(0, gpi.CopyData(ref resultPoints, ref resultTypes, startIndex, endIndex));
        }
    }

I am testing CopyData() from GraphicsPathIterator.

According GdipPathIterCopyData()

Expected is:

resultCount = 0.

But actual is:

System.ArgumentException : Parameter is not valid.

System.EntryPointNotFoundException : Unable to find an entry point named 'ZeroMemory' in DLL 'kernel32.dll'.

Satck Trace

[norek]

@mellinoe @hughbe I need to suspend my work for 3-4 weeks ;( untill I'll buy new computer.

[KostaVlev]

Because GraphicsPathIterator was short and I got it done. I started to work on PathGradientBrush. Then I ran into weird behavior of the SetSigmaBellShape() method and I need help.

The test method I wrote:

    [ConditionalTheory(nameof(PlatformDetection) + "." + nameof(PlatformDetection.IsNotWindowsNanoServer))]
    [InlineData(1f, 1f)]
    [InlineData(0f, 1f)]
    [InlineData(0.5f, 1f)]
    // The cases below fail, actual scale is always 1.
    [InlineData(1f, 0f)]
    [InlineData(0f, 0f)]
    [InlineData(0.5f, 0f)]
    [InlineData(1f, 0.5f)]
    [InlineData(0f, 0.5f)]
    [InlineData(0.5f, 0.5f)]
    public void SetSigmaBellShape_FocusScale_Succes(float focus, float scale)
    {
        using (PathGradientBrush brush = new PathGradientBrush(_defaultFloatPoints))
        {
            brush.SetSigmaBellShape(focus);
            Assert.True(brush.Transform.IsIdentity);
            if (focus == 0f)
            {
                Assert.Equal(256, brush.Blend.Positions.Length);
                Assert.Equal(256, brush.Blend.Factors.Length);
                Assert.Equal(focus, brush.Blend.Positions[0]);
                Assert.Equal(scale, brush.Blend.Factors[0]);
                Assert.Equal(1f, brush.Blend.Positions[brush.Blend.Positions.Length - 1]);
                Assert.Equal(0f, brush.Blend.Factors[brush.Blend.Factors.Length - 1]);
            }
            else if (focus == 1f)
            {
                Assert.Equal(256, brush.Blend.Positions.Length);
                Assert.Equal(256, brush.Blend.Factors.Length);
                Assert.Equal(0f, brush.Blend.Positions[0]);
                Assert.Equal(0f, brush.Blend.Factors[0]);
                Assert.Equal(focus, brush.Blend.Positions[brush.Blend.Positions.Length - 1]);
                Assert.Equal(scale, brush.Blend.Factors[brush.Blend.Factors.Length - 1]);
            }
            else
            {
                Assert.Equal(511, brush.Blend.Positions.Length);
                Assert.Equal(511, brush.Blend.Factors.Length);
                Assert.Equal(0f, brush.Blend.Positions[0]);
                Assert.Equal(0f, brush.Blend.Factors[0]);
                //Assert.Equal(1f, brush.Blend.Positions[16]);
                //Assert.Equal(0f, brush.Blend.Factors[16]);
                Assert.Equal(1f, brush.Blend.Positions[brush.Blend.Positions.Length - 1]);
                Assert.Equal(0f, brush.Blend.Factors[brush.Blend.Factors.Length - 1]);
            }
        }
    }

To write the test I was looking at GdipSetPathGradientSigmaBlend() .

Stack trace

Is the actual value the desired one?

[hughbe]

@KostaVlev interesting.

I have a little trouble here need somebody to confirm the following is correct.

This looks like an area where Wine doesn't have the same argument validation as GDI+ on Windows. It sounds like a Wine bug. GDI+ on Windows is the de-facto standard we should be validate our tests against, so if GDI+ on Windows gives a result that Wine/libgdiplus doesn't have, then that' going to be a bug in those open source implementations.

FWIW, libgdiplus seems to have the same behaviour: https://github.com/mono/libgdiplus/blob/1cbd58ef57c208a8517a772f3c8416169d7dfd9b/src/graphics-pathiterator.c#L120-L141. It could be that Windows used to allow this but changed behaviour and the open source libraries never found out or updated.

I'm runnning the following code against netfx locally, and it does not throw an EntrypointNotFoundException. This looks like a porting bug - I can reproduce the throwing behaviour with netcoreapp. I'm opening an issue for this

using System.Drawing;
using System.Drawing.Drawing2D;

namespace Test
{
    public class Program
    {
        public static void Main()
        {
            // Throw ArgumentException.
            //CopyData_StartEndIndexesOutOfRange_ReturnsExpeced(new PointF[3], new byte[3], -1, 2);
            //CopyData_StartEndIndexesOutOfRange_ReturnsExpeced(new PointF[3], new byte[3], 0, 3);

            // Return 0
            CopyData_StartEndIndexesOutOfRange_ReturnsExpeced(new PointF[3], new byte[3], int.MinValue, 2);
            CopyData_StartEndIndexesOutOfRange_ReturnsExpeced(new PointF[3], new byte[3], 0, int.MaxValue);
            CopyData_StartEndIndexesOutOfRange_ReturnsExpeced(new PointF[3], new byte[3], 2, 0);
        }

        public static void CopyData_StartEndIndexesOutOfRange_ReturnsExpeced(PointF[] points, byte[] types, int startIndex, int endIndex)
        {
            PointF[] resultPoints = new PointF[points.Length];
            byte[] resultTypes = new byte[points.Length];

            using (GraphicsPath gp = new GraphicsPath(points, types))
            using (GraphicsPathIterator gpi = new GraphicsPathIterator(gp))
            {
                gpi.CopyData(ref resultPoints, ref resultTypes, startIndex, endIndex);
            }
        }

    }
}

[hughbe]

Is the actual value the desired one?

I reckon this is another case where Wine and libgdiplus have different behaviour than Windows GDI - it's good to get these tests written to help us start bridging the gaps between the implementations.

I'm not 100% sure I understand what the actual and desired values are in this case - could you expand?

[KostaVlev]

    private readonly PointF[] _defaultFloatPoints = new PointF[2] { new PointF(1, 2), new PointF(20, 30) };

    [ConditionalTheory(nameof(PlatformDetection) + "." + nameof(PlatformDetection.IsNotWindowsNanoServer))]
    [InlineData(1f, 1f)]
    [InlineData(0f, 1f)]
    [InlineData(0.5f, 1f)]
    // The cases below fail, actual scale is always 1.
    //[InlineData(1f, 0f)]
    //[InlineData(0f, 0f)]
    //[InlineData(0.5f, 0f)]
    //[InlineData(1f, 0.5f)]
    //[InlineData(0f, 0.5f)]
    //[InlineData(0.5f, 0.5f)]
    public void SetSigmaBellShape_FocusScale_Succes(float focus, float scale)
    {
        using (PathGradientBrush brush = new PathGradientBrush(_defaultFloatPoints))
        {
            brush.SetSigmaBellShape(focus);
            Assert.True(brush.Transform.IsIdentity);
            if (focus == 0f)
            {
                Assert.Equal(256, brush.Blend.Positions.Length);
                Assert.Equal(256, brush.Blend.Factors.Length);
                Assert.Equal(focus, brush.Blend.Positions[0]);
                Assert.Equal(scale, brush.Blend.Factors[0]); // Here expected is as follow 1, 0, 0.5 but Actual is 1, 1, 1
                Assert.Equal(1f, brush.Blend.Positions[brush.Blend.Positions.Length - 1]);
                Assert.Equal(0f, brush.Blend.Factors[brush.Blend.Factors.Length - 1]);
            }
            else if (focus == 1f)
            {
                Assert.Equal(256, brush.Blend.Positions.Length);
                Assert.Equal(256, brush.Blend.Factors.Length);
                Assert.Equal(0f, brush.Blend.Positions[0]);
                Assert.Equal(0f, brush.Blend.Factors[0]);
                Assert.Equal(focus, brush.Blend.Positions[brush.Blend.Positions.Length - 1]);
                Assert.Equal(scale, brush.Blend.Factors[brush.Blend.Factors.Length - 1]);  // Here expected is as follow 1, 0, 0.5 but Actual is 1, 1, 1
            }
            else
            {
                Assert.Equal(511, brush.Blend.Positions.Length);
                Assert.Equal(511, brush.Blend.Factors.Length);
                Assert.Equal(0f, brush.Blend.Positions[0]);
                Assert.Equal(0f, brush.Blend.Factors[0]);
                //Assert.Equal(focus, brush.Blend.Positions[255]);
                //Assert.Equal(scale, brush.Blend.Factors[255]); // Here expected is as follow 1, 0, 0.5 but Actual is 1, 1, 1
                Assert.Equal(1f, brush.Blend.Positions[brush.Blend.Positions.Length - 1]);
                Assert.Equal(0f, brush.Blend.Factors[brush.Blend.Factors.Length - 1]);
            }
        }
    }

I also was looking here.

@hughbe I think is not setting correct the start and end points. I added comments above where Asserts fails.

[KostaVlev]

I need to clear this situation and I will be pretty much ready and with the PathGradientBrush tests.

I simplified the test I wrote:

      private readonly PointF[] _defaultFloatPoints = new PointF[2] { new PointF(1, 2), new PointF(20, 30) };

      public static IEnumerable<object[]> Blend_FactorsPositions_TestData()
    {
        yield return new object[] { new float[1] { 1 }, new float[1] { 0 } };
        yield return new object[] { new float[1] { 1 }, new float[3] { 0, 3, 1 } };
    }

    [ConditionalTheory(nameof(PlatformDetection) + "." + nameof(PlatformDetection.IsNotWindowsNanoServer))]
    [MemberData(nameof(Blend_FactorsPositions_TestData))]
    public void Blend_ReturnsExpected(float[] factors, float[] positions)
    {
        int expectedSize = factors.Length;

        using (PathGradientBrush brush = new PathGradientBrush(_defaultFloatPoints, WrapMode.TileFlipXY))
        {
            brush.Blend = new Blend { Factors = factors, Positions = positions };
            Assert.Equal(new float[1] { 0 }, brush.Blend.Positions);
        }
    }

Expected is: float[1] { 0 } Actual is: float[1]{random number} (well I think is random :)) number is in wide range. I got values from -3 to +7 running the test multiple times.

Not sure here if that inconstancy is acceptable.

The tested method

[mellinoe]

@KostaVlev It wasn't clear whether you are describing the behavior of the Windows implementation. That's the only one we are interested in validating -- any divergence from that by the other versions should be treated as a bug on those versions. If the Windows version is actually returning different values like you indicated, then the test needs to be changed (since it is asserting different behavior).

[KostaVlev]

@mellinoe I ran the test against netfx and returns expected 0 but running netcoreapp returns different numbers.

[hughbe]

Ohhh dear - can you make an issue?

[KostaVlev]

I figured out the questions I asked above. Seems like the entire time was just me :) ... I don't see anything free to take in System.Drawing.Drawing2D so I can start working in System.Drawing.Text

[mellinoe]

@KostaVlev Actually, we already have decent coverage of everything in System.Drawing.Text. Were you going to re-submit your tests for GraphicsPathIterator ? It is the last large type in Drawing2D that is untested. We could also use more test coverage for PathGradientBrush.

[KostaVlev]

@mellinoe I have the PathGradientBrush ready too. I will submitted it when we clear dotnet/corefx#22157

[mellinoe]

@KostaVlev Ah, great to hear. That will just about do it for Drawing2D, I believe.

[KostaVlev]

I don't see anybody working on System.Drawing.Imaging ImageAttributes and I think to take it...?

[hughbe]

@KostaVlev go ahead pal

[hughbe]

And thanks for jumping ahead and writing the tests. Something called life has got in the way of my contributions recently and @norek is unavailable too, so grab what you like (apart from Graphics.cs which I'm slowly working away on when I have time)

[hughbe]

I've just sent in a PR for BufferGraphics: https://github.com/dotnet/corefx/pull/22298

That's gonna be my last system.drawing pr for a couple of weeks so feel free to grab anything else including Graphics.cs which is the last big area missing tests

[KostaVlev]

I am trying to write test for ImageAttributes.SetOutputChannelColorProfile(). The test looks like this:

    [ConditionalFact(nameof(PlatformDetection) + "." + nameof(PlatformDetection.IsNotWindowsNanoServer))]
    public void SetOutputChannelColorProfile_Name_Success()
    {
        using (var bitmap = new Bitmap(_rectangle.Width, _rectangle.Height))
        using (var graphics = Graphics.FromImage(bitmap))
        using (var imageAttr = new ImageAttributes())
        {
            imageAttr.SetOutputChannel(ColorChannelFlag.ColorChannelC);
            imageAttr.SetOutputChannelColorProfile(Helpers.GetTestColorProfilePath("RSWOP.icm"));
            bitmap.SetPixel(0, 0, Color.FromArgb(255, 100, 100, 100));
            graphics.DrawImage(bitmap, _rectangle, _rectangle.X, _rectangle.Y, _rectangle.Width, _rectangle.Height, GraphicsUnit.Pixel, imageAttr);
            Assert.Equal(Color.FromArgb(255, 198, 198, 198), bitmap.GetPixel(0, 0));
        }
    }

I added hepper method GetTestColorProfilePath() and when I paste manually the RSWOP.icm file into F:\corefx\packages\system.drawing.common.testdata\1.0.2\content , the test works but how really do I add files to the packages? Am I allowed to do this?

[mellinoe]

@KostaVlev Please submit a PR to this repository, where the TestData packages are produced.

https://github.com/dotnet/corefx-testdata

All you need to do is add the file to the correct folder, and then update the nuspec: https://github.com/dotnet/corefx-testdata/blob/master/System.Drawing.Common.TestData.nuspec.

[mellinoe]

@KostaVlev I've updated a new version of System.Drawing.Common.TestData, version 1.0.3. You need to update this file in corefx in order to use the new version:

https://github.com/dotnet/corefx/blob/master/external/test-runtime/XUnit.Runtime.depproj#L77

You will also need to update the test project to point to the newer version: https://github.com/dotnet/corefx/blob/master/src/System.Drawing.Common/tests/System.Drawing.Common.Tests.csproj#L75

All of the parts that say 1.0.2 should be updated to 1.0.3. Bonus points if you define a property instead of repeating it, as well 😄

[KostaVlev]

@mellinoe Do you mean I can do something like

<PropertyGroup>
    <TestsDataVersion>1.0.3</TestsDataVersion>
</PropertyGroup>

in the System.Drawing.Common.Tests.csproj file?

[mellinoe]

@KostaVlev Yes, that's right. For now, though, just go ahead and hard-code the new version. I may actually want to create a property all the way up in https://github.com/dotnet/corefx/blob/master/dependencies.props, and use that property everywhere in the repo, including the test-runtime project file. That way it would only be specified once.

[KostaVlev]

Hi, the next one which isn't tested in Imaging is Metafile. I started working on it but :) I need a little bit help here.

I wrote this test:

[ConditionalFact(nameof(PlatformDetection) + "." + nameof(PlatformDetection.IsNotWindowsNanoServer))]
public void Ctor_IntPtrWmfPlaceableFileHeader_Success()
{
    using (var bufferGraphics = Graphics.FromHwndInternal(IntPtr.Zero))
    using (var metafile = new Metafile(bufferGraphics.GetHdc(), new WmfPlaceableFileHeader()))
    {
        Assert.Equal(new Rectangle(0, 0, 0, 0), metafile.GetMetafileHeader().Bounds);
    }
}

My idea is to create blank Metafile but I am getting System.Runtime.InteropServices.ExternalException : A generic error occurred in GDI+. when calling the constructor.

The exception occurs on every platform.

Any hints how to make this to work?

[mellinoe]

@KostaVlev Which call is throwing the exception? I'm pretty sure it is not valid to call FromHwndInternal using a null handle like that, so I'm not sure such a test is really valuable. A test could be added to assert that an ExternalException is thrown, but again, I don't think it is valuable, and I don't think we should care about that level of behavior compat to emulate it on the Unix version.

[KostaVlev]

@mellinoe The graphics object is created successfully var metafile = new Metafile(bufferGraphics.GetHdc(), new WmfPlaceableFileHeader()) throws the exception.

It tried this way too but still getting exception.

[ConditionalFact(nameof(PlatformDetection) + "." + nameof(PlatformDetection.IsNotWindowsNanoServer))]
 public void Ctor_IntPtrWmfPlaceableFileHeader_Success()
{
    using (var bitmap = new Bitmap(_rectangle.Width, _rectangle.Height))
    using (var bufferGraphics = Graphics.FromImage(bitmap))
    using (var metafile = new Metafile(bufferGraphics.GetHdc(), new WmfPlaceableFileHeader()))
    {
        Assert.Equal(new Rectangle(0, 0, 0, 0), metafile.GetMetafileHeader().Bounds);
    }
}

Since we have same behavior for netfx and netcoreapp should I skip this test for now?

[mellinoe]

Since we have same behavior for netfx and netcoreapp should I skip this test for now?

Yes

[KostaVlev]

I moved on System.Drawing.Printing.

[hughbe]

<3 good stuff

[mellinoe]

Code coverage numbers are looking really good to me (updated at the top), considering there are a few large files full of dead code (or code that doesn't really need to be tested). Thanks for all of the contributions, everyone!

[norek]

How nice, a lot of work was made since my "vacation". I just bought new computer and i'm ready to go!

[mellinoe]

Some more steady progress was made in the past month. I think @qmfrederik's current effort will give us enough coverage to fill in the remaining gaps and let us close this issue out.

[ghost]

Due to lack of recent activity, this issue has been marked as a candidate for backlog cleanup. It will be closed if no further activity occurs within 14 more days. Any new comment (by anyone, not necessarily the author) will undo this process.

This process is part of our issue cleanup automation.

[ghost]

This issue will now be closed since it had been marked no-recent-activity but received no further activity in the past 14 days. It is still possible to reopen or comment on the issue, but please note that the issue will be locked if it remains inactive for another 30 days.