Inspecting after a transform

[famschopman]

How do I inspect a dataset after a transform which changes the schema?

var categoricalEstimator = lContext.Transforms.Categorical.OneHotEncoding("StringProperty"); ITransformer categoricalTransformer = categoricalEstimator.Fit(dataView); IDataView transformedData = categoricalTransformer.Transform(dataView);

The following will fail because the schema has been updated with new vector types for the transformed dataset.

IEnumerable<MyModel> employeeDataEnumerable = mlContext.Data.CreateEnumerable<MyModel>(transformedData, reuseRowObject: true);

AB#1530397

---

Document Details

⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

• ID: 6c846fa2-eecd-aab0-3ebf-a52f5c2db94e
• Version Independent ID: fd80b6d4-04c0-34a3-09b3-d2d773eef9a3
• Content: Inspect intermediate data values during ML.NET processing - ML.NET
• Content Source: docs/machine-learning/how-to-guides/inspect-intermediate-data-ml-net.md
• Product: dotnet-ml
• GitHub Login: @luisquintanilla
• Microsoft Alias: luquinta

[luisquintanilla]

Hi @famschopman ,

In order to inspect it, you can create a new class that defines your transformed schema. Using your example:

// Original
public class MyModel
{
    public string StringProperty {get;set;}
}

// Transformed
public class MyTransformedModel
{
    public float StringProperty {get;set;}
}

var categoricalEstimator = lContext.Transforms.Categorical.OneHotEncoding("StringProperty");
ITransformer categoricalTransformer = categoricalEstimator.Fit(dataView);
IDataView transformedData = categoricalTransformer.Transform(dataView);

IEnumerable<MyTransformedModel> employeeDataEnumerable = mlContext.Data.CreateEnumerable<MyTransformedModel>(transformedData, reuseRowObject: true);

Notice how the type of your StringProperty has changed from string to float because now that column is the result of applying the OneHotEncoding transform.

[JRAlexander]

Hi, @famschopman! This appears resolved so we're going to close. Please reopen or comment here if that isn't the case. Thanks!