This lab will show you how to bring advanced ML vision capabilities to your applications with the Custom Vision Service. The service makes it easy to build custom image classifiers and provides APIs and tools to help you improve your classifier over time. The lab is split in two parts: first, we'll show you how to use the Custom Vision portal to build a classifier and then we'll show you how to do the same thing using the SDK. In both cases, we'll be building a classifier that can distinguish between a Surface Pro and a Surface Studio.
The Custom Vision portal requires either a work or school account (e.g. Office 365 accounts) or a personal Microsoft account (e.g. @outlook.com
). You can create a new personal Microsoft account here (signup.live.com) with an existing email address (or create a new one).
The first part of the lab, where the portal is used to build a classifier, only requires a modern web browser.
The second part of the lab, where the SDK is used to build a classifier, requires Visual Studio 2017 on Windows. Get it here. While the SDK in preview is Windows-only, the underlying RESTful APIs can be accessed from any platform. Learn more about the training API here and the prediction API here.
Before we can start training our classifier, we need to create a project in the Custom Vision portal. The Custom Vision Service is currently in preview so we don't need to provision a subscription in Azure (as is the case with most of the other Cognitive Services).
customvisionlab
Custom Vision Lab
General
(i.e. the default)Training an image classifier requires example images. While our data set is relatively small and can be uploaded via a web browser, it's likely that production scenarios will require hundreds or thousands of images and thus are best uploaded via the API (covered later in the lab).
$/Training/Images/surface-pro
and select all of the images.surface-pro
.$/Training/Images/surface-studio
and select all of the images.surface-studio
.Now that our data set is available, and each image correctly tagged, the training process can begin. This is a quick operation given the small size of our data set. The output is a classifier that can be used to make predictions. As more training data is added, we can create a new iteration of the classifier and analyze its performance.
It's recommended that at least 30 images of each class or category are included for a prototype classifier. In the interests of time, we're only providing 15 images for each class so we should expect slightly lower accuracy compared to a fully trained version.
The portal provides a test page for us to do a basic smoke test of our model with some new, unseen data. In a production scenario, the RESTful API is a more appropriate mechanism for doing bulk predictions (covered later in the lab).
$/Prediction/Images/surface-pro
and select the pro_test.jpg
image.surface-pro
.$/Prediction/Images/surface-studio
and select the studio_test.jpg
image.surface-studio
.Now that we've seen the Custom Vision Service in action, next we'll examine how we can use the API to complete the same process. Using the API helps us train and predict more efficiently. In order to accelerate the lab, we've prepared a basic console application as a starting point. In this lab, we'll be using the SDK - rather than the RESTful API - but the pattern of interaction is very similar.
$/CustomVision.sln
and wait for it to load.Microsoft.Cognitive.CustomVision.Training
in the search box and press Enter.The first step is to upload training data. In our case, the images are local so will be uploaded directly to the service; however, it's also possible to submit public URLs to training images and the service will automatically retrieve them. Once the images are uploaded, we can start the training process. This is an asynchronous operation so we need to monitor it.
Add the UploadTrainingData
method to upload the new training images to our project:
private static void UploadTrainingData(Guid projectId, TrainingApi trainingApi)
{
Console.WriteLine("Uploading images");
foreach (var tagName in new[] {"surface-pro", "surface-studio"})
{
var tag = trainingApi.CreateTag(projectId, tagName);
var images = Directory.GetFiles($@"..\..\Images\{tagName}")
.Select(f => new ImageFileCreateEntry(Path.GetFileName(f), File.ReadAllBytes(f)))
.ToList();
var batch = new ImageFileCreateBatch(images, new List<Guid> { tag.Id });
trainingApi.CreateImagesFromFiles(projectId, batch);
}
}
Add the TrainClassifier
method to invoke the training operation and wait for it to complete:
private static void TrainClassifier(Guid projectId, TrainingApi trainingApi)
{
Console.WriteLine("Training classifier");
var iteration = trainingApi.TrainProject(projectId);
while (iteration.Status == "Training")
{
Thread.Sleep(1000);
iteration = trainingApi.GetIteration(projectId, iteration.Id);
}
// Make the newly trained version the default for RESTful API requests
iteration.IsDefault = true;
trainingApi.UpdateIteration(projectId, iteration.Id, iteration);
Console.WriteLine("Training complete. Press any key to exit.");
}
Add the following code to the Main
method to retrieve credentials and invoke our new methods:
var trainingApi = new TrainingApi();
trainingApi.ApiKey = ConfigurationManager.AppSettings["trainingKey"];
var projectName = ConfigurationManager.AppSettings["projectName"];
var project = trainingApi.GetProjects().Single(p => p.Name == projectName);
UploadTrainingData(project.Id, trainingApi);
TrainClassifier(project.Id, trainingApi);
Console.ReadKey();
In order to interact with the training API, we need the key. This gives us access to all projects under our Custom Vision Service account. To identify the correct project to work with, we need the project name as well.
customvisionlab-api
Custom Vision Lab - API
General
(i.e. the default)<your_key>
with the key you copied to clipboard.
<add key="trainingKey" value="<your_key>" />
customvisionlab-api
).
<add key="projectName" value="<your_project>" />
The application will use the images under $\Training\Images\{tag}
in the training process. The entire process should only take a few moments given the small size of our data set.
customvisionlab-api
) and click the Performance tab.Now that the classifier is in place, we can use the prediction endpoint to classify new, unseen data. This can be accomplished with the prediction API.
Microsoft.Cognitive.CustomVision.Training
in the search box and press Enter.In our app, the training SDK is also required in order to resolve the project ID from the project name. The alternative would be to add the project ID to configuration and then this dependency can be eliminated.
Microsoft.Cognitive.CustomVision.Prediction
in the search box and press Enter.Add GetProjectId
method to resolve the project ID from the project name in our config file:
private static Guid GetProjectId()
{
var trainingApi = new TrainingApi();
trainingApi.ApiKey = ConfigurationManager.AppSettings["trainingKey"];
var projectName = ConfigurationManager.AppSettings["projectName"];
var project = trainingApi.GetProjects().Single(p => p.Name == projectName);
return project.Id;
}
GetPredictionEndpoint
method to retrieve credentials and prepare the SDK to make predictions:
private static PredictionEndpoint GetPredictionEndpoint()
{
var predictionEndpoint = new PredictionEndpoint();
predictionEndpoint.ApiKey = ConfigurationManager.AppSettings["predictionKey"];
return predictionEndpoint;
}
Add the following code to the Main
method to invoke the prediction endpoint:
var projectId = GetProjectId();
var endpoint = GetPredictionEndpoint();
Console.WriteLine("Testing images");
foreach (var tagName in new[] {"surface-pro", "surface-studio"})
{
var images = Directory.GetFiles($@"..\..\Images\{tagName}");
foreach (var image in images)
{
var memoryStream = new MemoryStream(File.ReadAllBytes(image));
var prediction = endpoint.PredictImage(projectId, memoryStream);
var topPrediction = prediction.Predictions.First();
Console.WriteLine($"{image} predicted to be {topPrediction.Tag} ({topPrediction.Probability:P1})");
}
}
Console.WriteLine("Press any key to exit");
Console.ReadKey();
In addition to the information required by the training API to resolve the project ID, a separate prediction API key is required to access the prediction functionality.
<your_key>
with the key from the Training project App.config.
<add key="trainingKey" value="<your_key>" />
<your_project>
with the project name from the Training project App.config.
<add key="projectName" value="<your_project>" />
<your_key>
with the key you copied to clipboard.
<add key="predictionKey" value="<your_key>" />
The application will use the images under $/Prediction/Images/{tag}
and return the most likely predictions for each.
pro_test.jpg
should be classified as a surface-pro
and the studio_test.jpg
should be classified as a surface-studio
.customvisionlab-api
) and click the Predictions tab.This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.microsoft.com.
When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.
This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.