Serialization in .Net 5.0

YnnamTenob commented 3 years ago

I have recently met with an issue where saving and serializating a model in .Net 5.0 (i.e. in .Net Notebooks or in Azure functions) is met with a Binary Serialization error. As far as I understand this error occurs due to the Security Concerns arround binary serialization. The recommended method of dealing with the Binary serailizer incompatibility is the implementation of the ISafeSerialization. After reviewing the code base in this repo that would require modifications all the way down to the Automaton to allow for safe serialization of models particularly for models that implement Incremental Learning hence need to be saved and serialized in the runtime environment. Is there a solution for this problem other than the obvious one I've mentioned above?

tminka commented 3 years ago

Try one of the other forms of serialization described at How to save distributions to disk.

YnnamTenob commented 3 years ago

Thank you very much @tminka

YnnamTenob commented 2 years ago

Hi @tminka I tried serializing using the Json Format as specified in the link you gave above. It serializes an object however said object cannot be deserialized.

The type that BayesPointMachineClassifier.CreateBinaryClassifier() creates is a:

CompoundBinaryStandardDataFormatBayesPointMachineClassifier<IList,Int32,.IList,Boolean>

However CompoundBinaryStandardDataFormatBayesPointMachineClassifier is internal to the library and cannot be deserialized, and as far as I know the interface that it implements also cannot be deserialized:

IBayesPointMachineClassifier<TInstanceSource, TInstance, TLabelSource, TStandardLabel, IDictionary<TStandardLabel, double>, TTrainingSettings, TPredictionSettings>

here is the json for the document that gets serialized. can you provide further guidance

tminka commented 2 years ago

How were you able to serialize that type using Json? It isn't designed to do that.

YnnamTenob commented 2 years ago

@tminka sorry for the delay I was out on PTO. here is what I did?

`using Net.Json; using Newtonsoft.Json; using Newtonsoft.Json.Serialization; using System.Collections.Concurrent; using Microsoft.ML.Probabilistic.Collections; using Microsoft.ML.Probabilistic.Learners.BayesPointMachineClassifierInternal;

var modelFile = "IdentifierSearchScoringModel.json";

class CollectionAsObjectResolver : DefaultContractResolver
{
private static readonly HashSet SerializeAsObjectTypes = new HashSet {
typeof(Vector), typeof(Matrix), typeof(IArray<>), typeof(ISparseList<>)
};

private static readonly ConcurrentDictionary<Type, JsonContract> ResolvedContracts = new ConcurrentDictionary<Type, JsonContract>();
public override JsonContract ResolveContract(Type type) => ResolvedContracts.GetOrAdd(type, this.ResolveContractInternal);
private JsonContract ResolveContractInternal(Type type) => IsExcludedType(type)? this.CreateObjectContract(type): this.CreateContract(type);
private static bool IsExcludedType(Type type) { if (type == null) return false; if (SerializeAsObjectTypes.Contains(type)) return true; if (type.IsGenericType && SerializeAsObjectTypes.Contains(type.GetGenericTypeDefinition())) return true; return IsExcludedType(type.BaseType) || type.GetInterfaces().Any(IsExcludedType);
} }

var serializerSettings = new JsonSerializerSettings {
TypeNameHandling = TypeNameHandling.Auto,
ContractResolver = new CollectionAsObjectResolver(),
PreserveReferencesHandling = PreserveReferencesHandling.Objects
};
var serializer = JsonSerializer.Create(serializerSettings);

var mapping = new ClassifierMapping();
var classifier = BayesPointMachineClassifier.CreateBinaryClassifier(mapping); classifier.Settings.Training.ComputeModelEvidence = true; // Train the Bayes Point Machine classifier classifier.Train(trainResult.features, trainResult.labels);

// write to disk
using (FileStream stream = new FileStream($"{simulationPath}/{dataVersion}/Engineered/{modelFile}", FileMode.Create))
{ var streamWriter = new StreamWriter(stream); var jsonWriter = new JsonTextWriter(streamWriter);
serializer.Serialize(jsonWriter, classifier);
jsonWriter.Flush();
}`

YnnamTenob commented 2 years ago

So if it isn't designed to do that how can I serialize a BPM model in .Net 5.0?

YnnamTenob commented 2 years ago

Hi @tminka,

Do you have any advice here? Not that it is any of your problem but I am coming up on a hard deadline and if I am not able to deserialize the model in .Net 5.0 I will have to scrap this approach.

Best, MB

tminka commented 2 years ago

The code that you sent doesn't work for me. As far as I can tell, Json.NET cannot serialize these classes at all. That is why I was confused how you got that output. The Learner classes all implement custom binary serialization. They would have to be changed to support any other form of serialization.

YnnamTenob commented 2 years ago

Thanks. I rrun that code in a .Net notebook on Azure Machine Learning. It serializes all of the child classes but does not serilize the top level class. One final question since I have the MeanTimePRecision and Precision can I use the generated algorithm to do inference.

tminka commented 2 years ago

Yes, if you are just making predictions then you can use the generated algorithm directly.

tminka commented 2 years ago

I have created PR #373 which adds the ability to serialize BayesPointMachineClassifiers as text.

YnnamTenob commented 2 years ago

@tminka Thank You. This is great.

YnnamTenob commented 2 years ago

HiI @tminka,

When will PR #373 make it to a released Nuget Package?

tminka commented 2 years ago

I will update the Nuget package this week.

dotnet / infer

Serialization in .Net 5.0 #371