dotnet / machinelearning

ML.NET is an open source and cross-platform machine learning framework for .NET.
https://dot.net/ml
MIT License
9.03k stars 1.89k forks source link

How should EntryPoint consumers register ML.NET assemblies? #2063

Closed montebhoover closed 5 years ago

montebhoover commented 5 years ago

@eerhardt, when we made HostEnvironmentBase and ComponentCatalog internal you mentioned that EntryPoint consumers like NimbusML or the GUI should be responsible for registering their ML.NET assemblies:

[2. ...

  1. Other subsystems (like the GUI, command-line, Entry Points, and model loading) will be responsible for registering the components they require in the manner they require.
  2. ...](https://github.com/dotnet/machinelearning/issues/208#issuecomment-422136134)

Could you give me some pointers on how to best go about that? Currently NimbusML registers assemblies using the ComponentCatalog from when it was public and we need to update this to ML.NET 0.9: https://github.com/Microsoft/NimbusML/blob/e1004720ec0c252ba87f02c190c33739d9c00f20/src/DotNetBridge/Bridge.cs#L314

cc: @TomFinley @yaeldekel @shmoradims

eerhardt commented 5 years ago

We may have gone too far internalizing things w.r.t. IHostEnvironment and ComponentCatalog.

As it currently sits, it is not easy to implement IHostEnvironment outside of ML.NET. This is because IHostEnvironment has a member:

        /// <summary>
        /// The catalog of loadable components (<see cref="LoadableClassAttribute"/>) that are available in this host.
        /// </summary>
        ComponentCatalog ComponentCatalog { get; }

However, ComponentCatalog has an internal constructor - meaning projects outside of ML.NET cannot instantiate it.

The one way to instantiate it publicly would be to create an MLContext, and use its ComponentCatalog instance.

So I guess it begs the same kinds of questions that @TomFinley is asking on #1959: Why does NimbusML need to implement IHostEnvironment and/or inherit from HostEnvironmentBase? Why can't it just use MLContext, like other public consumers do?

montebhoover commented 5 years ago

Thanks for the details. It sounds like MLContext may be sufficient after all. The two requirements for HostEnvironmentBase/MLContext in NimbusML are registering assemblies and getting access to an IChannel, and it looks like we can do both through MLContext.