search

dotnet / runtime

.NET is a cross-platform runtime for cloud, mobile, desktop, and IoT apps.
https://docs.microsoft.com/dotnet/core/
MIT License
14.91k stars 4.63k forks source link

Support Full System.Drawing Functionality on .NET Core #21980

Closed mellinoe closed 4 years ago

mellinoe commented 7 years ago

Update: The following is the current plan.

System.Drawing has been identified as one of the top most-used assemblies that are missing from .NET Core. Although there are 3rd-party alternatives that provide more modern versions of the same functionality (ImageSharp, SkiaSharp, etc.), we believe there is still value in providing a binary-compatible implementation of System.Drawing itself. This issue is intended to document and track the work items involved. There are still lots of open questions; feedback and suggestions are encouraged.

Priorities:

System.Drawing is, fundamentally, a very thin wrapper over the GDI+ component on Windows. Worse yet, it is not available for UWP applications, so support is limited strictly to Win32 applications. This makes cross-platform implementations difficult, and the largest chunk of work involved with this effort will be figuring out how we can support the library on all of our platforms, and maintaining compatibility between them. There are several existing implementations that we can leverage, but none will support all of our platforms by themselves. We may need several distinct backends in order to get full cross-platform support, although some options could avoid that. The below chart maps implementation approaches (rows) with platform applicability (columns):

|Win32 | UWP | macOS | Linux / Others -----|--------|-------|---------|--------- .NET Framework Version (GDI+) | ✓ | X | X | X Direct2D/Win2D|✓|✓|X|X Mono/libgdiplus|✓|X|✓|✓ Xamarin/CoreGraphics|X|X|✓|X Managed|✓|✓|✓|✓ ### .NET Framework Version (GDI+) This is the existing implementation from the .NET Framework on Windows. It is a very thin wrapper around GDI+. * :heavy_check_mark: Baseline implementation. Well-tested, stable. * :x: Can only be used in non-UWP Windows apps (Win32). ### Direct2D/Win2D In order to get UWP support, we could implement a compatibility layer over Direct2D, or Win2D. * :heavy_check_mark: GPU acceleration * :x: Only buys us UWP support over the existing .NET Framework implementation. * :x: High implementation cost * Can be shared between Win32 and UWP applications (Windows 8+ only). ### Mono/libgdiplus This is the implementation used in mono. It uses a complicated native shim called "libgdiplus", which relies on a large set of native binaries. * :heavy_check_mark: libgdiplus is available in a large number of package managers across the Unix ecosystem already. This means we can treat the dependency the same way we do the other native dependencies for .NET Core. * :x: Native code adds a lot of complexity for new devs. ### Xamarin/CoreGraphics This is a specialized implementation used by Xamarin, in support of macOS and iOS. Unlike the "libgdiplus" mono version, it only uses CoreGraphics (a system component). * :heavy_check_mark: No extra native dependencies; only uses system libraries. * :x: macOS-specific. Could be used for iOS (not supported by .NET Core yet). ### Managed (ImageSharp or custom) One option is to implement the library in managed code. We can call into an existing library, like ImageSharp, for image-processing and graphics functionality. This is very desirable for a lot of reasons, but it does have a higher implementation cost. * :heavy_check_mark: Can be used everywhere. * :heavy_check_mark: Can achieve uniform behavior everywhere (at the expense of conformity with .NET Framework). * :heavy_check_mark: Lower maintenance costs in the long-term. * :x: High implementation cost. ## Windows-Specific One option is to treat the library as a Windows-specific component, similar to Registry. Attempting to install and load the library on other platforms would throw PlatformNotSupportedException's, as Registry does. ## Tests We do not have a very good existing test suite in the .NET Framework. The library is mainly covered indirectly, through other components like WinForms. Mono has a good set of conformance tests for their implementation, which we should be able to re-use. Where appropriate, we should also consider test cases which directly compare output from the .NET Framework implementation and other implementations (assuming we choose to support such things). ## Location An open question is where this library should live. If we take third-party dependencies to support some platforms, we may not want this code to live in corefx. ## Printing Functionality One thing skimmed over above is the fact that System.Drawing exposes a lot of functionality for printing images and documents. This poses a different problem from image-processing and graphics, which can be implemented portably. We may decide to exclude all printing-related portions of the library. ## Future investments Once the library is supported on all platforms, we will accept small, targeted improvements to the library, keeping in mind that legacy compatibility is the only important metric for the library. We will not want to merge any code that refactors, optimizes, or augments the Windows version until we have a fully-working cross-platform implementation. This matches our strategy with other compatibility libraries, like System.IO.SerialPort. @marek-safar @jimbobsquarepants @ViktorHofer @qmfrederik @karelz
qmfrederik commented 7 years ago

@mellinoe Thanks for the additional information!

I actually have a good experience with using Mono/libgdiplus via CoreCompat.System.Drawing. Here's why:

Another option would be to use the System.Drawing code from NetFX and have it consume libgdiplus instead of GDI+ on Linux. There's been some talk of open sourcing System.Drawing before, not sure if that's still an option? It would be fairly easy to implement, too.

Regarding UWP, do you currently have demand for System.Drawing on UWP?

/cc @akoeplinger

mellinoe commented 7 years ago

All Linux distros which .NET Core supports include libgdiplus in their mainline repositories: Ubuntu, Debian, Fedora, OpenSuse - even Alpine has it. Plus, libgidplus was recently added to Homebrew, too.

That is good to know; thanks for the info. If we are able to treat it as a fully external dependency, like the other native libs we use, then I would have less hesitation about using it. I'm not an expert on libgdiplus, so I would certainly like to understand more about it. The folks that I have talked with have been hesitant to readily suggest that we go ahead and use it, but it may be okay in the face of less-promising alternatives. In general, though, I think we should be really cautious about any new native dependencies we adopt in .NET Core. They make a lot of things more complicated for us and for customers.

Could you elaborate on "Native code adds a lot of complexity for new devs"?

I was trying to express the idea that folks trying to submit fixes for System.Drawing will have a harder time contributing to an external C library than a managed C# library here. Just a different skill-set, is all.

There's been some talk of open sourcing System.Drawing before, not sure if that's still an option? It would be fairly easy to implement, too.

At the very least, we will be using the .NET Framework version for Windows, so we will certainly open-source it. We can attempt to use that same implementation against libgdiplus on other platforms, but I fear that would involve reworking tons of the native code. Then again, that is just a guess on my part.

Regarding UWP, do you currently have demand for System.Drawing on UWP?

We view System.Drawing as one of the main libraries blocking old customers (and their code) from migrating to our new platforms, including UWP. That's not exactly the question you asked, but the end result is: we'd like to support System.Drawing everywhere, because it has proven to be an important dependency for existing libraries.

qmfrederik commented 7 years ago

At the very least, we will be using the .NET Framework version for Windows, so we will certainly open-source it. We can attempt to use that same implementation against libgdiplus on other platforms, but I fear that would involve reworking tons of the native code. Then again, that is just a guess on my part.

My guess is that the .NET Framework version will work well with libgdiplus. It's an assumption based on my experience with libgdiplus and how it tries to be binary compatible with GDI+. The proof of the pudding is in the eating, though. If you can make the .NET Framework System.Drawing source code available under an open source license, I'd be happy to give it a spin, and run Mono's unit tests for System.Drawing on .NET Framework's System.Drawing backed by libgdiplus running on .NET Core on, say, Linux -- that would give the definitive answer.

I was trying to express the idea that folks trying to submit fixes for System.Drawing will have a harder time contributing to an external C library than a managed C# library here. Just a different skill-set, is all.

That's a fair point, although on the other hand the C libraries that libgdiplus uses - such as libpng, libjpeg,... have matured for a long time so there's some value to that, too. Graphics code requires a special skillset anyway, perhaps the bulk of people with that skillset are knowledgeable in C?

Perhaps another option is to start with libgdiplus and move code from unmanaged to managed. It could help kick start the System.Drawing implementation by reusing libgdiplus and move towards the goal of cross-platform managed code step by step.

JimBobSquarePants commented 7 years ago

Perhaps another option is to start with libgdiplus and move code from unmanaged to managed. It could help kick start the System.Drawing implementation by reusing libgdiplus and move towards the goal of cross-platform managed code step by step.

If I was to build a managed 2D graphics library (I am) I wouldn't, by choice, re-implement the System.Drawing API.

That said @mellinoe I'm happy to offer and assist in migration if the decision was made to use ImageSharp as an underlying managed library. In Jamestopia though we wouldn't bother and would actively push developers to migrate to better API's. Seems a lot cheaper and easier for MS.

There are some fundamental differences in how the two libraries currently work which would have to be discussed and workarounds considered. We're still unfinished so we have great flexibility to alter the API to improve flexibility and fill gaps which would help a lot.

Pixel Formats ImageSharp expands all image formats into one of 20 different pixel formats we bundle. System.Drawing stores images in Format8bppIndexed format and similar in their indexed form. There would have to be a workaround developed to do the same.

Image Formats

MetaData System.Drawing bundles everything together into the PropertyItem class from EXIF metadata to gif loop count and frame delay and makes no effort to decode that data. We've split out metadata into generic, EXIF, and ICC components and offer full reading/writing for those data types. You would have to create Property items to match that data and store the raw bytes.

Affine Transforms - Resize, Crop etc We would need to do some work to ensure that we can produce the same output as System.Drawing. I'm not sure what sampler it uses for the different quality enumerations. We also don't support matrix operations within resize operations so there would have to be work done there.

Drawing We offer drawing with a very similar API to System.Drawing. There may be a few gaps but I imagine. I know off-hand we haven't implemented all the Hatch bushes @tocsoft is our resident expert there so he can fill you in on what is/isn't there.

I'm sure there's more...

Regarding "Behavior and performance issues", I'm not really aware of them, so it would be good to have them listed, too.

@qmfrederik I had a very, very quick scan of the libgdi codebase and immediately found unsupported areas 1, 2. I've no idea what is out there. I remember having issues with Image.FromStream(stream, useembeddedColorManagement) in the past but I don't know if that is still an issue.

Graphics code requires a special skillset anyway, perhaps the bulk of people with that skillset are knowledgeable in C?

Btw I couldn't read/write C if you held a gun to my head plus I'm teaching myself graphics programming as I write ImagSharp. Shocking 😝

migueldeicaza commented 7 years ago

It is worth updating the original issue with the new information provided in the comments.

Mono's System.Drawing implementation was used for Mono's Windows.Forms implementation and we spent a few years tuning it to work with various third party component vendors. We delegated the hard graphic problems to Cairo.

It was also built so that on Windows, it would use the system provided GDI+ library, and on other platforms, it uses Mono's libgdiplus, it achieves this by having libgdiplus surface the same API as the Windows libgdiplus.

This means a couple of things:

While the implementation works, the font handling capabilities are complicated on Unix, so we used FreeType to get this information. While this works great on Linux systems, it is very slow on first use on a Mac that typically lacks FreeType, so if I was going to do the work in this space, I would start by improving libgdiplus to have a swappable font backend and use Apple's CoreText on OSX to avoid the initial font cache creation problems.

Cairo and the underlying libraries are faster than any C# code you can write.

That said, if I was going to take on the long-term maintenance of System.Drawing, today I would take a different approach.

I would use SkiaSharp as the backing engine as it is actively maintained, actively developed, actively fine tuned and optimized for both Chrome and Android. It also addresses some of the design limitations of Cairo, both the bottlenecks of the API as well as issues with the coordinate space that happen due to loss of precision with huge matrix transformations (Mozilla went through this, and made essentially this switch as well).

I would start with the sysdrawing-coregraphics which is a project that moved most of the logic from libgdiplus from C to C# and replaced the calls to Cairo with calls to CoreGraphics.

And then I would replace the calls to CoreGraphics with calls to the Skia APIs.

Last year I did some work along those lines on my spare time, where I mostly butchered the codebase so it would compile without CoreGraphics and I was planning on adding the Skia support. The time to do that never materialized. If you are interested in that fork, you can find it here.

JimBobSquarePants commented 7 years ago

Just had a thought. What about server support. What are the plans there? That should factor into any decision making.

TheBlueSky commented 7 years ago

So what is the goal here? Only API compatibility? This is what I understood from what @mellinoe said.

If this is the case, @migueldeicaza comparison between native code and C# isn't very relevant, as long as C# performance is decent:

Cairo and the underlying libraries are faster than any C# code you can write.

I mean if building the API in managed code is an option, let's go for it.

On the other hand, if we settled on "let's just create something" option, let's not go with Win32-only option; this makes it a bit confusing.

vibeeshan025 commented 7 years ago

Great initiative!! Its better to have System.Drawing implemented on top of set of interfaces and let the community develop each implementation. Example we can use DirectX on windows and OpenGL on linux and mac OS. but the System.Drawing will internally call the interfaces which is platform agnostic. If we are going for windows only option then its not going to make any difference to existing .net frameworks.

JimBobSquarePants commented 7 years ago

You need good APIs for that that to be a viable idea. This is a stopgap not a solution.

bitapparat commented 7 years ago

I'd personally prefer a more forward-looking approach with an officially supported modern cross-platform graphics API for all .Net platforms, and a generic and optional System.Drawing implementation on top of it. But i'm aware that's a rather extensive and expensive solution and, thus, probably unrealistic.

gulbanana commented 7 years ago

I would love to see something like an implementation on top of ImageSharp. This hypothetical System.Drawing port isn't for high performance or high quality use - it's just to remove a blocker for porting legacy code in which image handling might be one small function. It would be fine if it was slower than the original netfx System.Drawing or didn't produce bit-identical output in all ops- it might even be able to get away with throwing NotSupported for some rarely-used apis.

Building on top of a community library with a modern API would provide a path forward for people to move their code to make direct use of that implentation later on.

stefanolson commented 7 years ago

Whilst I'm aware that a lot of people use System.Drawing, the WPF System.Windows.Media api is a lot more capable in many areas, and more similar to the UWP API so personally I would rather see a better, more flexible/modern API than System.Drawing available, although of course there may be valuable things in System.Drawing that may be worth implementing

Rekeyea commented 7 years ago

IMHO System.Drawing should be left as Windows only. It is already being used in .Net Framework projects using ASP.Net Core, but those will still be using Windows since those are enterprise applications and I don't see them moving their whole infrastructure "because now they can".

ViktorHofer commented 7 years ago

IMHO System.Drawing should be left as Windows only. It is already being used in .Net Framework projects using ASP.Net Core, but those will still be using Windows since those are enterprise applications and I don't see them moving their whole infrastructure "because now they can".

I disagree. ASP.NET Core is being used by more and more open source projects and startups because of cross-platform support and its speed improvements in comparison to nodeJS. I think it's viable to support System.Drawing on all environments where .NET Core and ASP.NET Core is running.

I'm a little worried about binary compatibility here. I think that we definitely need a System.Drawing implementation which is compatible with .NET Framework but I also think that we should build something (together with the community) which is modern, scalable and fast. It might be a good idea to 1. port the existing lib to core (managed) and 2. build a successor of System.Drawing which has all the innovations and improvements which aren't possible with the current API spec.

Rekeyea commented 7 years ago

@ViktorHofer it was said that there are other more modern alternatives like ImageSharp or SkiaSharp. That's why I think it's pointless since those open source projects and startups should use those other alternatives.

JimBobSquarePants commented 7 years ago

@ViktorHofer I've literally spent the last two years of my life doing just that with ImageSharp. It's already faster than System.Drawing in many respects and has a much simpler/easier to extend API.

It needs the community to step up to help me push it over the line. Building something else would be a crushing blow to me.

ViktorHofer commented 7 years ago

@JimBobSquarePants Then we should definitely also talk about building upon existing libraries 👍 If you put a lot of effort into it we shouldn't exclude ImageSharp from the discussion.

benaadams commented 7 years ago

What's the goal?

  1. Is it so people can move their apps from Framework to Core with minimal changes?
  2. Is it so people can move their apps from Framework on Windows to Core on Linux?
  3. Is it so people can develop new server apps with an api that is remarked as:

    Classes within the System.Drawing namespace are not supported for use within a Windows or ASP.NET service. Attempting to use these classes from within one of these application types may produce unexpected problems, such as diminished service performance and run-time exceptions.

I had initially imagined it was for (1) where people used System.Drawing in spite of the warnings; but maybe is for (2) and (3)?

willdean commented 7 years ago

@JimBobSquarePants What needs to be done to remove the "in early stages (alpha). As such, we cannot support its use on production environments until the library reaches release candidate status." warning from your project?

It's rather off-putting, and given the way that the .NET Core team have redefined 'Release Candidate' to mean what 'Alpha Experimental Technology Preview' might have once meant, it's now anyone's guess in the world of .NET what "early stages (alpha)" might mean, but it does sound like "not something I want to invest time in right now".

If you want to mop-up the world of System.Drawing users - I'm one, with a important (to us) ASP.Net application which can't currently be ported to .NET Core because it depends on, inter alia, System.Drawing - then you perhaps need to work a little on the optics for that market.

I don't want to see anyone suffer 'a crushing blow', especially an unintended one from a myopic pachyderm like MS, but I went to look at ImageSharp this morning and came away discouraged.

Kukkimonsuta commented 7 years ago

I'd go with path of least resistance - identify which scenario would be the cheapest and do that. Put warnings all around that it's just compatibility layer that should not be used for new projects. If feeling bad about leftover manpower invest it in improving ImageSharp :)

qmfrederik commented 7 years ago

@willdaen Out of curiosity, did you have a look at CoreCompat.System.Drawing (Mono's Sysyem.Drawing ported to Core)? Would that meet your requirements?

qmfrederik commented 7 years ago

@Kukkimonsuta Yeah, that kind of is what I thought this was about - a cross platform impkementation of System.Drawing which people can (temporarly) use as they move to .NET Core - basically removing one of the adoption blockers.

It may not be the most pure solution, but it kind of is what libgdiplus was built for.

JimBobSquarePants commented 7 years ago

@willdean

I went to look at ImageSharp this morning and came away discouraged.

Really? Well that's terribly disappointing.

You didn't see the feature list linked to from the readme? I need to update that actually to add all our Porter Duff support and flesh out some other things.

Can you make it clear that it works with full .NET, rather than people needing to pick through the platform types?

It's in the readme.

Built against .Net Standard 1.1 ImageSharp can be used in device, cloud, and embedded/IoT scenarios

The message means exactly what it says. The API is subject to change. (Btw I don't think MS have redefined anything, they just rushed things certainly but alpha, beta etc is not defined by them)

What needs to be done to remove the "in early stages (alpha). As such, we cannot support its use on production environments until the library reaches release candidate status." warning from your project?

We need to do two things.

  1. Get the jpeg decoder refactored to reduce memory usage and speed it up.
  2. Change the IPixel interface to not pack/unpack from bytes and use predefined structs instead. RGBA32, RGB24, BGRA32, BGR24 and also use refs for the Vector4 packing/unpacking

After that I'm happy to move to beta and add the library to Nuget.

It's been a very slow alpha as I want to get this right. There's simply no point creating a half/baked graphics library. No one will use it and you'll end up wasting your time.

P.S. I've no shame in admitting I learned by doing. It's the absolute best way to thoroughly understand things IMO and I see it as an encouragement that others can do the same. 😄

willdean commented 7 years ago

@JimBobSquarePants Thanks for the response - I am really reluctant to hijack this thread by extending discussion about ImageSharp specifically... I'll give it a try and file any issues on your repo.

willdean commented 7 years ago

@qmfrederik I haven't looked at CoreCompat.System.Drawing - we're blocked on several things (SignalR is another) and I haven't really searched hard for alternatives to everything which is missing - I just keep a vague eye on what's happening to the things I know will cause us problems.

karelz commented 7 years ago

Lots of great feedback from anyone, thank you! (I didn't catch up on all of it)

Here's how I look at System.Drawing priorities and options:

Priorities

  1. Primary goal of System.Drawing is to simplify migration from Desktop to .NET Core (on Windows), i.e. high-compat between Desktop and Core implementation on Windows.
  2. Second-level goal is to support System.Drawing where it is needed -- currently Mono needs it on Mac & Linux (Mono is trying to reuse CoreFX libraries / source code, so synergy is desired).
    • Note: Mono has additional requirement on iOS -- size has to be small (i.e. it requires using Mac APIs - @marek-safar to confirm if I didn't mix it up with our discussion about HttpClient).
  3. Third-level (stretch) goal is to simplify migration from Windows to Linux/Mac on .NET Core.
    • Note: Easy migration does not necessarily imply great perf.
  4. Non-goals are high-perf, rich OS-specific capabilities, modern API surface, etc. (anything not listed in (1)-(3)).

Solutions

(2) Mono needs it on Mac & Linux

(3) simplify migration from Windows to Linux/Mac

qmfrederik commented 7 years ago

@karelz Regarding the Mono implementation on Linux, I don't think it is involved at all. In fact, libgdiplus, the native part, ships with all Linux distros .NET Core supports. So you don't need to port it, the distros have already done that for you. You can declare a dependency on libgdiplus pretty much the same way you do for openssl or curl - acquire it via the distro instead of NuGet.

You can consider distributing libgdiplus and its dependencies as native components un NuGet packages. That does get complicated, fast - I tried with CoreCompat.System.Drawing.

Mono/libgdiplus gets you 1, 2 and 3 at almost no cost - most of the work has already been done. The result is in the CoreCompat.System.Drawing NuGet package.

karelz commented 7 years ago

@qmfrederik I was relying only on info from @marek-safar who works on Mono - so I let him chime in.

migueldeicaza commented 7 years ago

Mr @qmfrederik is correct, libgdiplus is available on pretty much every Linux distribution already and it only relies on the unmanaged pieces. If Marek had any feedback on dealing with build complexity, it would be around you trying to ship a forked version of yours, and then you would have the same issues that you have in shipping CoreCLR - namely that you will want to compile for each system.

So that should not be an issue if you want to take the fastest/easiest path to bring System.Drawing.

You do not need to worry about iOS, for historical reasons System.Drawing is not used there and we do not even distribute it (beyond the basic Rectangle, Point types).

On iOS, the world has gone with either CoreGraphics or our cross-platform SkiaSharp.

Given the outlined priorities from @karelz, I would change my recommendation. Given these priorities, really System.Drawing wont be a long-term foundation for graphics, and I would essentially just reuse the Mono code.

Rely on gdiplus, and use Mono's System.Drawing stack (and ifdef out parts that wont work on .NET Core 2).

You can certainly choose a different path, but given the above goals, it sounds like investing on rewriting this code is a waste of time.

karelz commented 7 years ago

Given these priorities, really System.Drawing wont be a long-term foundation for graphics, and I would essentially just reuse the Mono code.

That was my thinking as well. With the caveat that we might want to use Desktop implementation on Windows to have higher-confidence compat (I've been bitten by HttpListener Mono compat recently, so I'm a bit cautious now ;)).

given the above goals, it sounds like investing on rewriting this code is a waste of time.

Agreed. We didn't plan huge investment. And I am not fan of wasting effort.

Overall, I don't think System.Drawing is such a great API & implementation that we want to innovate on it -- other projects like SkiaSharp, ImageSharp already did that, so let's rely on them to innovate for future in the graphics space. We should not reinvent the wheel in CoreFX, unless there are really good reasons. One .NET ecosystem to rule them all! ;)

migueldeicaza commented 7 years ago

Mono's System.Drawing approach has the advantage that it calls into the native GDI+, so on Windows, you get to use the one that ships with the system already. And if we get approval for release of .NET's System.Drawing, you can swap that out as well.

(I've been bitten by HttpListener Mono compat recently, so I'm a bit cautious now ;)).

The feeling is mutual @karelz, Mono has also been bitten by code in CoreFX :-)

karelz commented 7 years ago

I agree that more code/library sharing with Mono would be beneficial overall. If we can address the compat concern on Windows (likely with lots of tests), I'd be fine with that approach :)

The feeling is mutual @karelz, Mono has also been bitten by code in CoreFX :-)

When you hit those cases (or if you have a list from the past), please send the feedback to us/me. While it is usually not actionable right away, it is critical to prevent such unfortunate situations in future. Without knowing about such cases, we won't fully understand the impact we have on Mono and we will likely keep doing what we're doing, causing you the same pain over and over again. Let's try to prevent that.

qmfrederik commented 7 years ago

What would the next steps be? Building a NuGet package for .NET Core from Mono's sources is straightforward, as should be setting up CI and tests. Would Mono produce the package or would the code be exported to a separate repository under dotnet? There's also the work of ingesting the NetFX code if possible.

migueldeicaza commented 7 years ago

@karelz What part of the compat with Windows are you concerned about? Like I said, our System.Drawing is a thin layer on top of GDI+, so on Windows, it is the Windows GDI+ that does all the heavy lifting.

There are likely gaps in our implementation (both on the managed and unmanaged side) and those should be filled.

Perhaps the most comprehensive test that we have is Mono's Windows.Forms implementation, which emulates the entire toolkit (even on Windows) on top of pure GDI+. There are two other test suites, a unit testing suite that is already in place that tests some elements of GDI+ and one that we had for fuzzy-comparison of image outputs, this latter test suite needs to be brought back to life.

Logistically speaking, I would like to keep the same source tree compiling on both Mono and .NET Core to maximize code sharing, and avoid forks as this is a single unit.

Let us start by saying that we should add the proper ifdefs to make the source code compile both against .NET Core and Mono.

akoeplinger commented 7 years ago

@migueldeicaza Mono's System.Drawing+libgdiplus already compiles and works on .NET Core due to some great PRs that @qmfrederik sent over the last week/months :)

I agree that open sourcing the managed System.Drawing layer from .NET Framework would be great so we can replace our managed layer and be more compatible (I'm assuming that most of the work is done by the underlying gdiplus in the .NET Framework implementation too).

This would mean that on Windows you'd end up with a highly compatible version and other platforms have Mono's libgdiplus underneath. Moving to a separate repo or into corefx sounds good to me.

And of course we should promote ImageSharp etc as the modern alternative!

mellinoe commented 7 years ago

Thanks for all the great feedback, everyone. I'll try to summarize the main points discussed, and also try to clear up a couple of things.

qmfrederik commented 7 years ago

@mellinoe If you want to give things a try, have a look at the code here: https://github.com/corecompat/system.drawing. It contains a .NET Core 2.0 solution which compiles Mono's System.Drawing and runs the unit tests, and has CI support (Travis for Linux and AppVeyor for Windows)

MaximRouiller commented 7 years ago

Whatever happens, make sure it 100% works in Azure. GDI+ is highly restricted right now and most PDF libraries depend on this through System.Drawing

qmfrederik commented 7 years ago

@JimBobSquarePants I own a couple of NuGet packages (SharpAdbClient, RemoteViewing, CoreCompat.Selenium) which somewhere in their public API return images (screenshots, really). These packages don't process the image -- it's the consumers of these packages that will do image processing.

Currently, they depend on System.Drawing and return an Image object. That creates a hard dependency on System.Drawing, something I don't necessarily want to do.

So I'm trying to figure out what I (and I guess others which encounter the same use case) can do to make sure my packages don't have a hard dependency on System.Drawing, and let my consumers decide how to process the images.

The only alternatives I can think of right now is

Thoughts?

JimBobSquarePants commented 7 years ago

@qmfrederik

I think something like @karelz described - a compat pack would be the best idea that would live in my repo. We'd bridge Image via loading/saving from streams and common primitives also.

vibeeshan025 commented 7 years ago

You guys are doing a great job. Keep it up. One thing I understood by reading this thread is it's not going to be easy to port system.drawing for .Net core and preserving cross compatibility.

Since .Net core is already not compatible with existing .Net framework, what if we have a minimal implemention of an API from scratch. Which wraps around platform specific APIs or a common cross platform library. (Cairo) My major concern is we should not lose the cross compatibility of .Net core for features. I am not advocating that we should not implement against system.drawing, but we should not lose the cross compatibility for the sake of legacy code compatibility.( But it's still great if we can achieve both)

Again good job .Net core team.

benaadams commented 7 years ago

we should not lose the cross compatibility for the sake of legacy code compatibility

@vibeeshan025 My understanding is System.Drawing is being provided for legacy code compatibility and it is a legacy api. So compatibility is the high bar. For new code alternative apis are be encouraged. Its not available on UWP and it is explicitly discouraged for certain applications:

Classes within the System.Drawing namespace are not supported for use within a Windows or ASP.NET service. Attempting to use these classes from within one of these application types may produce unexpected problems, such as diminished service performance and run-time exceptions.

clrjunkie commented 7 years ago

@mellinoe very clear summary! 👍

Managed (ImageSharp or custom)

One option is to implement the library in managed code. We can call into an existing library, like ImageSharp, for image-processing and graphics functionality. This is very desirable for a lot of reasons, but it does have a higher implementation cost.

Voting for this option. I bet image resizing and cropping are the 80%. I assume the implementation for both is purely computational and does not depend on any particular O/S specific API and even if there is some GPU accelerated version I bet there is a non-accelerated counterpart. Perhaps there is an option to extract the relevant native implementation code from Windows use it to create a small cross-platform library. Open sourcing it would nice but don’t think many would care so long it works..

marek-safar commented 7 years ago

@karelz @mellinoe I think it'd be better to open source .NET System.Drawing and use that on netcore and Mono on Windows. That'd bring 100% compatibility with no effort.

On linux my concern was that you'll fall into same trap as you did with OpenSSL dependency which might not be that bad if you the library is there for backward compatibility solution only.

OSX has been mostly ignored here but having libgdiplus dependency there is certainly not smooth experience as it brings whole X11 (not part of OSX anymore) dependency making self-contained app build tricky.

qmfrederik commented 7 years ago

@marek-safar Regarding macOS & x11, you can now build libgdiplus for macOS without X11 and acquire libgdiplus via Homebrew.

Deployment of libgdiplus would be similar to the OpenSSL dependency, with all its pros and cons.

marek-safar commented 7 years ago

@qmfrederik right but that disables X11 features which at quick check looks will break DPI scaling.

qmfrederik commented 7 years ago

@marek-safar Yup, (automatic) DPI scaling will not work with libgdiplus on macOS. I assumed that you'd only really need DPI scaling when using WinForms or another UI on the Mac itself; and that this wouldn't be much (if any) of a problem when using System.Drawing in ASP.NET Core or CLI tools?

shanselman commented 7 years ago

@JimBobSquarePants

JimBobSquarePants commented 7 years ago

So where are we on this?

Using libgdiplus looked promising for a while but there's two glaring issues as far as I can see.

  1. Missing or broken features in Mono System.Drawing
  2. Using the library on a server.

We've all read the warning but have continued to ignored it so far. We cannot possible recommend to migrating developers that it's ok to use the compat pack on a server surely?

karelz commented 7 years ago

@qmfrederik Deployment of libgdiplus would be similar to the OpenSSL dependency, with all its pros and cons.

We got rid of OpenSSL dependency on Mac, mostly due to strong feedback from Mac users. I don't want to follow the same route (depending on Homebrew-installed dependency, which Homebrew can break). If Mac code needs something that is NOT part of the OS, we should not make it part of CoreFX to avoid another nightmare and being "different"/"special". (feel free to call me out if I don't get all the nuances - my primary goal is to optimize for simplicity, complexity usually bites back)

Overall: It seems to me that there is no simple solution for Mac. If that's correct, I would suggest to rely on 3rd-party libraries shims (see https://github.com/dotnet/corefx/issues/20325#issuecomment-304315828).

@marek-safar wht did you mean by Linux having same problems as OpenSSL? Overall we do not mind that too much on Linux AFAIK (on Mac it was much worse). @janvorli can you comment on our position towards OpenSSL dependency on Linux? I think we made our peace with it, right? What are the biggest cons and how should we think about adding yet-another-dependency like that?