MissingMethodException: Method not found: '!!0 TorchSharp.ModuleExtensionMethods.cuda(!!0, Int32)'

[dansimon363]

Visual Studio 2022 - Version 17.10.4 x64 TorchSharp-cpu (0.102.7)

Using the following code causes an exception in TorchSharp-cpu version 0.102.7 but works correctly in version 0.101.5.

Error occurs on this line: ITransformer mlModel = mlContext.Model.Load(mlNetModelPath, out var _);

using Microsoft.ML;
using Microsoft.ML.Data;
using Microsoft.ML.TorchSharp;
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;

public static IOrderedEnumerable<KeyValuePair<string, float>> Predict(string sentimentText, string outputModelPath)
{
    ModelInput modelInput = new ModelInput()
    {
        SentimentText = sentimentText
    };

    Lazy<PredictionEngine<ModelInput, ModelOutput>> predictEngine = new Lazy<PredictionEngine<ModelInput, ModelOutput>>(() => CreatePredictEngine(outputModelPath), true);
    var predEngine = predictEngine.Value;
    var output = predEngine.Predict(modelInput);
    var scores = output.Score;
}

private static PredictionEngine<ModelInput, ModelOutput> CreatePredictEngine(string mlNetModelPath)
{
    var mlContext = new MLContext();
    ITransformer mlModel = mlContext.Model.Load(mlNetModelPath, out var _);
    return mlContext.Model.CreatePredictionEngine<ModelInput, ModelOutput>(mlModel);
}

private class ModelInput
{
    [LoadColumn(0)]
    [ColumnName(@"Sentiment")]
    public float Sentiment { get; set; }

    [LoadColumn(1)]
    [ColumnName(@"SentimentText")]
    public string SentimentText { get; set; }
}

private class ModelOutput
{
    [ColumnName(@"Sentiment")]
    public uint Sentiment { get; set; }

    [ColumnName(@"SentimentText")]
    public string SentimentText { get; set; }

    [ColumnName(@"PredictedLabel")]
    public float PredictedLabel { get; set; }

    [ColumnName(@"Score")]
    public float[] Score { get; set; }
}

System.Reflection.TargetInvocationException
  HResult=0x80131604
  Message=Exception has been thrown by the target of an invocation.
  Source=System.Private.CoreLib
  StackTrace:
   at System.Reflection.MethodBaseInvoker.InvokeDirectByRefWithFewArgs(Object obj, Span`1 copyOfArgs, BindingFlags invokeAttr)
   at System.Reflection.MethodBaseInvoker.InvokeWithFewArgs(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at System.Reflection.MethodBase.Invoke(Object obj, Object[] parameters)
   at Microsoft.ML.Runtime.ComponentCatalog.LoadableClassInfo.CreateInstanceCore(Object[] ctorArgs)
   at Microsoft.ML.Runtime.ComponentCatalog.LoadableClassInfo.CreateInstance(IHostEnvironment env, Object args, Object[] extra)
   at Microsoft.ML.Runtime.ComponentCatalog.TryCreateInstance[TRes](IHostEnvironment env, Type signatureType, TRes& result, String name, String options, Object[] extra)
   at Microsoft.ML.Runtime.ComponentCatalog.TryCreateInstance[TRes,TSig](IHostEnvironment env, TRes& result, String name, String options, Object[] extra)
   at Microsoft.ML.ModelLoadContext.TryLoadModelCore[TRes,TSig](IHostEnvironment env, TRes& result, Object[] extra)
   at Microsoft.ML.ModelLoadContext.TryLoadModel[TRes,TSig](IHostEnvironment env, TRes& result, RepositoryReader rep, Entry ent, String dir, Object[] extra)
   at Microsoft.ML.ModelLoadContext.LoadModel[TRes,TSig](IHostEnvironment env, TRes& result, RepositoryReader rep, Entry ent, String dir, Object[] extra)
   at Microsoft.ML.ModelLoadContext.LoadModelOrNull[TRes,TSig](IHostEnvironment env, TRes& result, RepositoryReader rep, String dir, Object[] extra)
   at Microsoft.ML.ModelLoadContext.LoadModel[TRes,TSig](IHostEnvironment env, TRes& result, RepositoryReader rep, String dir, Object[] extra)
   at Microsoft.ML.ModelOperationsCatalog.Load(Stream stream, DataViewSchema& inputSchema)
   at Microsoft.ML.ModelOperationsCatalog.Load(String filePath, DataViewSchema& inputSchema)
   at SingularAgent.Helpers.TextClassificationHelper.CreatePredictEngine(String mlNetModelPath) in C:\Users\danth\Files\Projects\SingularAgent\SingularAgent\Helpers\TextClassificationHelper.cs:line 129
   at SingularAgent.Helpers.TextClassificationHelper.<>c__DisplayClass9_0.<Predict>b__0() in C:\Users\danth\Files\Projects\SingularAgent\SingularAgent\Helpers\TextClassificationHelper.cs:line 112
   at System.Lazy`1.ViaFactory(LazyThreadSafetyMode mode)

  This exception was originally thrown at this call stack:
    [External Code]

Inner Exception 1:
TargetInvocationException: Exception has been thrown by the target of an invocation.

Inner Exception 2:
MissingMethodException: Method not found: '!!0 TorchSharp.ModuleExtensionMethods.cuda(!!0, Int32)'.

I was informed of a workaround via this post: https://stackoverflow.com/questions/78630239/missingmethodexception-method-not-found-0-torchsharp-moduleextensionmethods.

[NiklasGustafsson]

Yes, if you are using TorchSharp with ML.NET, you do have to use the right version. TorchSharp is still in preview, so breaking binary compatibility is still possible.

[dansimon363]

Ok, thanks for confirming.

[NiklasGustafsson]

Should the issue be closed, or is there an action item here?

[dansimon363]

I'm not sure if it should be closed. I'll defer to your expertise.

If this is expected, then it would be nice if there was some way that a dev could know about breaking binary compatibilities when using TorchSharp and ML.NET (if there isn't a way to already know). I think I made the assumption that using the latest version of packages for everything would work. I don't know if that would be appropriate in RELEASENOTES.md or README.md or somewhere else (if at all). I understand that being in preview can cause breaking changes so this is just a suggestion.

[NiklasGustafsson]

When you make a reference to ML.NET in your project, it should pull in the right version of TorchSharp. Is that not what you're seeing? If 0.102.7 is pulled in by ML.NET, I would think that's something that has to be fixed in ML.NET's package dependency definitions.

[dansimon363]

When I add a reference to Microsoft.ML.TorchSharp, it pulls in TorchSharp version 0.101.5.

I must have added TorchSharp-cpu reference at some point while trying to get the different types of ML .NET examples working in my project and then updated all the packages.

[NiklasGustafsson]

So, it seems it's behaving like it should.

The problem you ran into is that we are not guaranteeing binary compatibility between releases, so when another package takes a dependency on TorchSharp, it is to a specific version of it.

The update to cuda() was to enable asynchronous moves, which aligns with PyTorch. Since ML.NET pulls in the right version, it seems to me that there's no further action to be taken (besides moving TorchSharp beyond preview status :-))

[dansimon363]

Thanks for the explanation @NiklasGustafsson.