search

accord-net / framework

Machine learning, computer vision, statistics and general scientific computing for .NET
http://accord-framework.net
GNU Lesser General Public License v2.1
4.49k stars 1.99k forks source link

.NET Core version #318

Closed zeroskyx closed 7 years ago

zeroskyx commented 8 years ago

Greetings,

are there any plans to provide a .NET Core version (netstandard) for this framework in the near future? This would allow new applications such as running on very different platforms.

Thanks in advance and best regards -Simon

cesarsouza commented 8 years ago

@Hello Simon,

Right now the project is supported on .NET core through the Portable Accord.NET packages, which are currently provided by the CureOS Shim project, but officially supported by both projects.

Hope it helps!

Best regards, Cesar

zeroskyx commented 8 years ago

Hi Cesar,

thank you very much for your fast response! I am able to get portable.accord.vision to run on the full .NET framework and on UWP. Unfortunately, the PCL does not support the (new) netstandard (https://docs.microsoft.com/en-us/dotnet/articles/standard/library) for .NET Core that would allow us to use it on any .NET platform (even Mac and Linux) and on ASP.NET Core 1.0.

The following is the respective output from the package manager:

PATH=.\node_modules.bin;C:\Program Files (x86)\Microsoft Visual Studio 14.0\Web\External;%PATH%;C:\Program Files (x86)\Microsoft Visual Studio 14.0\Web\External\git C:\Program Files\dotnet\dotnet.exe restore "C:\Users\Simon\OneDrive\Visual Studio\Prototypes\ConsoleBase.vs\restore.dg" log : Restoring packages for C:\Users\Simon\OneDrive\Visual Studio\Prototypes\ConsoleBase\ConsoleBase\project.json... log : Installing shim 2.1.0-pre. log : Installing portable.accord.core 3.1.0-pre. log : Installing portable.accord.math 3.1.0-pre. log : Installing portable.accord.statistics 3.1.0-pre. log : Installing portable.accord.machinelearning 3.1.0-pre. log : Installing portable.accord.imaging 3.1.0-pre. log : Installing portable.accord.vision 3.1.0-pre. error: Package portable.accord.vision 3.1.0-pre is not compatible with netcoreapp1.0 (.NETCoreApp,Version=v1.0). Package portable.accord.vision 3.1.0-pre supports: portable-net45+win8+wpa81 (.NETPortable,Version=v0.0,Profile=Profile111) error: Package portable.accord.imaging 3.1.0-pre is not compatible with netcoreapp1.0 (.NETCoreApp,Version=v1.0). Package portable.accord.imaging 3.1.0-pre supports: portable-net45+win8+wpa81 (.NETPortable,Version=v0.0,Profile=Profile111) error: Package shim.drawing 3.0.0 is not compatible with netcoreapp1.0 (.NETCoreApp,Version=v1.0). Package shim.drawing 3.0.0 supports: error: - monoandroid10 (MonoAndroid,Version=v1.0) error: - monotouch10 (MonoTouch,Version=v1.0) error: - net45 (.NETFramework,Version=v4.5) error: - portable-net45+win8+wpa81 (.NETPortable,Version=v0.0,Profile=Profile111) error: - portable-win81+wpa81 (.NETPortable,Version=v0.0,Profile=Profile32) error: - xamarinios10 (Xamarin.iOS,Version=v1.0) error: Package portable.accord.machinelearning 3.1.0-pre is not compatible with netcoreapp1.0 (.NETCoreApp,Version=v1.0). Package portable.accord.machinelearning 3.1.0-pre supports: portable-net45+win8+wpa81 (.NETPortable,Version=v0.0,Profile=Profile111) error: Package portable.accord.statistics 3.1.0-pre is not compatible with netcoreapp1.0 (.NETCoreApp,Version=v1.0). Package portable.accord.statistics 3.1.0-pre supports: portable-net45+win8+wpa81 (.NETPortable,Version=v0.0,Profile=Profile111) error: Package portable.accord.math 3.1.0-pre is not compatible with netcoreapp1.0 (.NETCoreApp,Version=v1.0). Package portable.accord.math 3.1.0-pre supports: portable-net45+win8+wpa81 (.NETPortable,Version=v0.0,Profile=Profile111) error: Package portable.accord.core 3.1.0-pre is not compatible with netcoreapp1.0 (.NETCoreApp,Version=v1.0). Package portable.accord.core 3.1.0-pre supports: error: - net45 (.NETFramework,Version=v4.5) error: - netcore451 (.NETCore,Version=v4.5.1) error: - portable-net45+win8+wpa81 (.NETPortable,Version=v0.0,Profile=Profile111) error: Package shim 2.1.0-pre is not compatible with netcoreapp1.0 (.NETCoreApp,Version=v1.0). Package shim 2.1.0-pre supports: error: - monoandroid10 (MonoAndroid,Version=v1.0) error: - monotouch10 (MonoTouch,Version=v1.0) error: - net40 (.NETFramework,Version=v4.0) error: - netcore45 (.NETCore,Version=v4.5) error: - netcore451 (.NETCore,Version=v4.5.1) error: - portable-net40+sl5+win8+wp8+wpa81 (.NETPortable,Version=v0.0,Profile=Profile328) error: - portable-net45+win8+wp8+wpa81 (.NETPortable,Version=v0.0,Profile=Profile259) error: - portable-net45+win8+wpa81 (.NETPortable,Version=v0.0,Profile=Profile111) error: - portable-win81+wpa81 (.NETPortable,Version=v0.0,Profile=Profile32) error: - sl5 (Silverlight,Version=v5.0) error: - uap (UAP,Version=v0.0) error: - wp8 (WindowsPhone,Version=v8.0) error: - xamarinios10 (Xamarin.iOS,Version=v1.0) error: One or more packages are incompatible with .NETCoreApp,Version=v1.0. log : Writing lock file to disk. Path: C:\Users\Simon\OneDrive\Visual Studio\Prototypes\ConsoleBase\ConsoleBase\project.lock.json log : C:\Users\Simon\OneDrive\Visual Studio\Prototypes\ConsoleBase\ConsoleBase\ConsoleBase.xproj log : Restore failed in 11994ms. Errors in C:\Users\Simon\OneDrive\Visual Studio\Prototypes\ConsoleBase\ConsoleBase\ConsoleBase.xproj Package portable.accord.vision 3.1.0-pre is not compatible with netcoreapp1.0 (.NETCoreApp,Version=v1.0). Package portable.accord.vision 3.1.0-pre supports: portable-net45+win8+wpa81 (.NETPortable,Version=v0.0,Profile=Profile111) Package portable.accord.imaging 3.1.0-pre is not compatible with netcoreapp1.0 (.NETCoreApp,Version=v1.0). Package portable.accord.imaging 3.1.0-pre supports: portable-net45+win8+wpa81 (.NETPortable,Version=v0.0,Profile=Profile111) Package shim.drawing 3.0.0 is not compatible with netcoreapp1.0 (.NETCoreApp,Version=v1.0). Package shim.drawing 3.0.0 supports:

I tried the last pre-version (https://www.nuget.org/packages/portable.accord.vision/3.1.0-pre).

Best regards -Simon

anders9ustafsson commented 8 years ago

Hi guys,

no, I would not expect the portable libraries to work out-of-the-box on .NET Core, some adaptations are required. In particular, libraries with imaging functionality are not applicable in .NET Core. On the other hand, the portable libraries should at least support UWP already today.

I hope to get time to fully adapt the portable libraries to .NET Core, but as it is right now I am a little short of time.

Regards, Anders @ Cureos

anders9ustafsson commented 8 years ago

Hi again,

I wanted to add that with .NET Core the need for a portable subset of Accord might not even be warranted, at least for the non-imaging libraries. I have made extremely basic tests on AForge.Core, and it is possible to build it practically without changing any code and without referencing the Shim library.

For example, there are .NET Core compatible NuGet libraries coming up for binary serialization (currently in pre-release), which will make building considerably easier.

I'll keep you updated with my progress on this.

Regards, Anders @ Cureos

cesarsouza commented 8 years ago

Hi Anders, Simon,

My apologies. I was under the impression that .NET core was quite similar to UWP and the Shim would work for it. I guess I have to catch up a bit with the differences between the latest .NET flavors before making such statements.

Anders: I have been thinking for a while now about removing Accord.Imaging's dependence on System.Drawing and creating a separate Accord.Imaging.WinForms (or similar) to keep existing functionality. Do you think this would make things easier for you, or do you think it would be better to do this solely from the Shim side?

Regards, Cesar

anders9ustafsson commented 8 years ago

I have been thinking for a while now about removing Accord.Imaging's dependence on System.Drawing and creating a separate Accord.Imaging.WinForms (or similar) to keep existing functionality. Do you think this would make things easier for you, or do you think it would be better to do this solely from the Shim side?

@cesarsouza I think that would be a splendid idea! If you could make the (core) imaging library platform agnostic, it would be much easier to "plug-in" rendering adapters for various platforms on top of that.

I have done similar things in a few other packages, in particular fo-dicom and CSJ2K, so at least theoretically I can help out, although I am a little bit short of time for the moment.

cesarsouza commented 8 years ago

It seems I can base Accord.Imaging on System.Drawing.Primitives and UnmanagedImage instead. Since all filters are written to handle UnmanagedImage at their core, it shouldn't take much effort. Overloads for Bitmap can be offered by the .WinForms binary through extension methods, which should avoid breaking compatibility too much.

anders9ustafsson commented 8 years ago

I ran a quick .NET Standard 2.0 compatibility test with the ApiPort tool on the 3.3.0 .NET 4.5 Release binaries, and here is the verdict:

accord-portability

Please note that .NET Standard 2.0 is not yet released! I do not know the exact time frame for public availability. .NET Standard 1.x is considerably less compatible overall.

Of course, the controls and imaging based libraries are not expected to be highly compatible, since System.Drawing and System.Windows.Forms assemblies are evidently not part of the .NET Standard. ADO.NET is also not part of .NET Standard, which should explain why compatibility in Math, Statistics and IO libraries are also relatively low.

I am enclosing the full report for detailed information of API incompatibilities.

ApiPortAnalysis(1).xlsx

UPDATE OCT 17 Please be aware that the compatibility results for the audio projects are misleading. The third party SharpDX libraries which are an essential part have not been considered in the review.

cesarsouza commented 8 years ago

Thanks a lot Anders, this is ultra useful. On a side note, I couldn't use System.Drawing.Primitives as I initially planned. Even after adding the reference through NuGet, it seems the compiler cannot see the types exposed by the library. Even when trying to inspect it in the object browser I cannot see any types.

Do you perhaps know if there is anything extra that has to be done in order to reference a .NET standard/core library in a full .NET 4.6 project?

Regarding incompatibility with ADO, we could just remove methods that rely on them with some preprocessor directives. Their presence should not be critical to any of the assemblies in project.

anders9ustafsson commented 8 years ago

Do you perhaps know if there is anything extra that has to be done in order to reference a .NET standard/core library in a full .NET 4.6 project?

I haven't too much experience with it myself yet, but from what I know referencing a .NET Standard library in a .NET 4.5 and higher project is fairly straightforward, pretty much like referencing a PCL project.

.NET Standard 2.0 will be applicable to .NET Framework 4.6.1 and higher, see here.

cmaart commented 7 years ago

+1

dbruning commented 7 years ago

From the .NET Core roadmap: 2.0 Preview (of .NET Standard 2.0 and .NET Core 2.0) should be out soon (Q2 2017) Then 2.0 full release should be Q3 2017

I'd love to be able to use Accord.NET from a UWP project. I tried the portable.accord library but got stuck on a variant of this issue. I'm going to wait until UWP and Accord.NET are both on .NET Standard 2.0, in the hope that by removing some of the hoops that portable.accord is jumping through, that native compilation issue goes away too.

sophisma commented 7 years ago

Hi guys, I'm trying Xamarin Forms and Accord for the first time so, I'm sorry if this is very basic. I just created a Xamarin Forms blank project and updated it to NetStandard. When I try to install the portable.accord.machinelearning NuGet on the Common Project (the one referenced by all the specific platform projects) I'm getting this:

Package portable.accord.machinelearning 3.0.2.60302 is not compatible with netstandard1.4 (.NETStandard,Version=v1.4). Package portable.accord.machinelearning 3.0.2.60302 supports: portable-net45+win8+wpa81 (.NETPortable,Version=v0.0,Profile=Profile111) Package portable.accord.statistics 3.0.2.60302 is not compatible with netstandard1.4 (.NETStandard,Version=v1.4). Package portable.accord.statistics 3.0.2.60302 supports: portable-net45+win8+wpa81 (.NETPortable,Version=v0.0,Profile=Profile111) Package portable.accord.math 3.0.2.60302 is not compatible with netstandard1.4 (.NETStandard,Version=v1.4). Package portable.accord.math 3.0.2.60302 supports: portable-net45+win8+wpa81 (.NETPortable,Version=v0.0,Profile=Profile111) Package portable.accord.core 3.0.2.60302 is not compatible with netstandard1.4 (.NETStandard,Version=v1.4). Package portable.accord.core 3.0.2.60302 supports:

Is there anything that can be done to install this on a NetStandard Project?

vpenades commented 7 years ago

Regarding Net.Standard compatible imaging libraries (for interoperability), there's two very promising libraries... both compatible with Net.Standard 1.3:

ImageSharp is a pure image processing library, it's in it's early development stages but worth following.

SkiaSharp is the actual rendering engine of Google Chrome, freed as open source and made fully cross platform, adopted by xamarin to be the backend rendering system for xamarin.forms.

Both provide very complete Bitmap processing features.

anders9ustafsson commented 7 years ago

An even simpler approach is to use the CoreCompat System.Drawing package for .NET Standard 1.3 and higher, which is fully compatible with the System.Drawing namespace. Using this package, Accord.NET image processing code would be applicable without modification on .NET Core.

anders9ustafsson commented 7 years ago

UPDATED MAY 15

Since .NET Core and Standard 2.0 Preview 1 was released last week, I'd thought I should check the compatibility of some of the Accord class libraries.

Here are my results so far, using preprocessor directive NET46 where relevant:

Assembly Builds? Comment
Accord.Core Yes
Accord.DataSets Yes Referencing .NET Framework 2.0 NuGet package SharpZipLib
Accord.Fuzzy Yes
Accord.Genetics Yes
Accord.Imaging Yes System.Drawing namespace provided by .NET Standard 2.0 NuGet package CoreCompat.System.Drawing.v2
Accord.IO Yes Need to exclude ExcelReader, since System.Data.OleDb namespace is not included in .NET Standard 2.0
Accord.MachineLearning Yes Need to exclude System.Reflection.Emit.AssemblyBuilder based DecisionTree.ToAssembly methods
Accord.MachineLearning.GPL Yes
Accord.Math.Core Yes
Accord.Math Yes
Accord.Math.NonCommercial Yes
Accord.Neuro Yes
Accord.Statistics Yes
Accord.Text Yes
Accord.Video Yes
Accord.Video.DirectShow Yes Need to exclude Windows Forms based control VideoCaptureDeviceForm
Accord.Video.Kinect Yes
Accord.Video.VFW Yes
Accord.Video.Ximea Yes
Accord.Vision Yes

So in summary, libraries that do not include explicit Windows Forms and/or sound functionality can be built practically without modification for .NET Standard 2.0.

System.Drawing based packages rely on the System.Drawing for .NET Core NuGet package that targets .NET Standard 2.0. It should be noted that the device specific video libraries rely on P/Invoke calls and might therefore only be functional on Windows.

For those feeling inclined to test .NET Standard 2.0 compatibility themselves, the adopted project files are available in respective project folder in my personal fork of Accord.NET Framework.

cesarsouza commented 7 years ago

Amazing! Thanks Anders!

I've started creating a new solution file for the project to target .NET Standard 2.0.

Regards, Cesar

cesarsouza commented 7 years ago

Hi everyone,

I have just committed a new solution file (named Accord.NET (NETStandard).sln) that can be used to build a version of the framework for .NET Standard 2.0. However, I haven't been able to run the unit tests yet to verify that everything is indeed working. The problem is that the unit tests projects currently target NUnit 2, and it seems that there are no NUnit 2 packages for .NET Core or .NET Standard.

As such, I guess the next step would be to resolve Issue #407, porting the existing unit tests to the newer fluent syntax used in NUnit 3. I will probably start this task by tomorrow.

Regards, Cesar

anders9ustafsson commented 7 years ago

@cesarsouza In my ".NET Standard sandbox fork" I made some attempts with unit tests as well. The issues I could identify so far (Core, Math, Statistics, Machine Learning and IO tested) when moving to NUnit 3 are these:

There is replacement source code for ExpectedExceptionAttribute here, although I believe the natural way to move forward would be to use Assert.Throws instead, or Assert.Catch if you additionally want to validate the exception message.

I have not figured out a sufficient replacement for Timeout yet. There is still the MaxTimeAttribute although it only checks that the test completes within a certain timeframe, it does not terminate the test when timed out.

cesarsouza commented 7 years ago

Thanks @anders9ustafsson!

Actually I thought this was going to be much harder. I thought it was mandatory to convert all Assert's and others to their fluent versions (i.e Assert.That, etc). I've just committed a version of the unit test projects that currently compile under .NET Standard 2.0.

However, I wasn't able to actually run them yet. It seems there is no NUnit Test Runner for VS2017 / .NETStandard yet (https://github.com/nunit/nunit3-vs-adapter/issues/305). I've tried to implement some of the workarounds listed in that issue page (installing NUnit Test Runner 3.8.0-preview, adding full debug to the test assemblies), but I couldn't make much progress.

I will try again tomorrow.

Regards, Cesar

anders9ustafsson commented 7 years ago

@cesarsouza It was not entirely straightforward, but what I did was to create a console application and add a specific test runner to each unit test library. I don't remember right now where I got the code from, but you can look at the test runner here. It relies on NUnitLite, so if you have not added that package already, you will need to (in parallel with NUnit).

It's not a very flexible solution, I am afraid. I have not found any way to run only a sub-set of the tests yet, for example.

anders9ustafsson commented 7 years ago

@cesarsouza BTW, have you been able to work your way around the ExcelReader limitation? There is a NuGet package ExcelDataReader that targets netstandard 1.3 and later, which I suppose could be a good replacement for accessing Excel files via OleDb. Their Github page is here.

cesarsouza commented 7 years ago

For the moment I am not including ExcelReader in the .NET Standard release, but I can add it back later. Thanks a lot for the ExcelDataReader suggestion @anders9ustafsson

By the way, I have just uploaded some pre-release (alpha) netstandard packages to NuGet. I haven't tested any of them yet, though.

cesarsouza commented 7 years ago

Added in release 3.6.0.

vpenades commented 7 years ago

Great!, it's nice to see things are moving in the right direction!

Now it's a matter of waiting for Net.Core, Xamarin and UWP to catch up with net.standard 2.0, which I guess will happen by the end of the year.

ConductedClever commented 7 years ago

Hi. Using dot net core 1.1 DataTable is not accessible. How can I use the algorithms? I explained the issue here https://stackoverflow.com/questions/45729504/asp-core-1-1-datatable-exists-in-both. TG.

cesarsouza commented 7 years ago

Hi @ConductedClever,

DataTable is not available in .NET Core 1.1, but it is in .NET Core 2.0. If you can upgrade your project to .NET Core 2.0, then you will be able to use it in your code.

However, if you cannot switch to .NET Core 2.0, then please note that you are not required to use DataTables with any of the methods in the framework. They are given or shown just because they can give some extra convenience, but they are not really required.

For example, please take a look at the example for learning a Naive Bayes classifier in the documentation for the NaiveBayesLearning class. All methods in the framework can be trained from jagged arrays, such as int[][], double[][] or string[][] instead of datatable.

In fact, even the codification filter (which is used to transform strings into numbers such that they can be processed by the learning algorithms) can also work with jagged arrays of strings (if you would need this functionality).

Hope it helps, Cesar

vincenzoml commented 6 years ago

Hi there. I'm starting to port an image processing research prototyppe to fsharp on .net core (previously, we were using OCaml). Currently I'm using simpleITK dlls, which is far from the best way to go. I'd love to use accord-net instead. I'm a bit confused, is this the right package to test?

https://www.nuget.org/packages/portable.accord.imaging/3.1.0-pre

TheBricktop commented 4 years ago

will this enable usage of accord net on android? or linux?