Need a way to predict a batch of examples in one call to Tensorflow

[marcmm]

System information

• OS version/distro: Windows 10
• .NET Version (eg., dotnet --info): .Net Core 2.1

Issue

• What did you do? Using Tensorflow extension, there's no support to predict a batch of examples in one call to Tensorflow.
• What happened? This functionality is not supported at this time.
• What did you expect? It would be good to support this functionality to improve efficiency in model prediction.

Source code / logs

Please paste or attach the code or logs or traces that would be helpful to diagnose the issue you are reporting.

[najeeb-kazmi]

@marcmm you can do this currently, see this sample.

https://github.com/dotnet/machinelearning/blob/71f0dc28939681198d99c2eb86e1ed326cc8c7d5/docs/samples/Microsoft.ML.Samples/Dynamic/TensorFlow/ImageClassification.cs#L37-L44

Note line 44 which does estimator.Transform(idv). idv can be an IDataView containing many examples. The call to Transform() would predict on all of them.

This other sample shows single predictions by creating a PredictionEngine. However, in place of line 123, you could simply do var batchPredictions = model.Transform(dataView), with dataView being an IDataView containing multiple examples. https://github.com/dotnet/machinelearning/blob/71f0dc28939681198d99c2eb86e1ed326cc8c7d5/docs/samples/Microsoft.ML.Samples/Dynamic/TensorFlow/TextClassification.cs#L63-L64 https://github.com/dotnet/machinelearning/blob/71f0dc28939681198d99c2eb86e1ed326cc8c7d5/docs/samples/Microsoft.ML.Samples/Dynamic/TensorFlow/TextClassification.cs#L99-L123