IBM / powerai-counting-cars

Run a Jupyter Notebook to detect, track, and count cars in a video using Maximo Visual Insights (formerly PowerAI Vision) and OpenCV
https://developer.ibm.com/patterns/detect-track-and-count-cars-in-a-video/
Apache License 2.0
327 stars 176 forks source link
computer-vision ibmcode jupyter-notebook mp4-video opencv powerai powerai-vision video

WARNING: This repository is no longer maintained

This repository will not be updated. The repository will be kept available in read-only mode.

Detect, track, and count cars in a video

video-to-gif

Note: This repo has been updated to use Maximo Visual Inspection (formerly known as PowerAI Vision). Everything is the same except for the name itself.

Whether you are counting cars on a road or products on a conveyer belt, there are many use cases for computer vision with video. With video as input, automatic labeling can be used to create a better classifier with less manual effort. This code pattern shows you how to create and use a classifier to identify objects in motion and then track the objects and count them as they enter designated regions of interest.

In this code pattern, we will create a video car counter using Maximo Visual Inspection, OpenCV and a Jupyter Notebook. We'll use a little manual labeling and a lot of automatic labeling to train an object classifier to recognize cars on a highway. We'll load another car video into a Jupyter Notebook where we'll process the individual frames and annotate the video.

We'll use our deployed model for inference to detect cars on a sample of the frames at a regular interval. We'll use OpenCV to track the cars from frame to frame in between inference. In addition to counting the cars as they are detected, we'll also count them as they cross a "finish line" for each lane and show cars per second.

Credit goes to Michael Hollinger for his initial notebook counting objects with PowerAI Vision.

When the reader has completed this code pattern, they will understand how to:

architecture

Flow

  1. Upload a video using the Maximo Visual Inspection web UI.
  2. Use automatic labeling and train a model.
  3. Deploy the model to create a Maximo Visual Inspection (formerly known as PowerAI Vision) inference API.
  4. Use a Jupyter Notebook to detect, track, and count cars in a video.

Included components

Featured technologies

Watch the Video

video

Prerequisites

PowerAI Vision

This code pattern requires Maximo Visual Inspection (formerly known as PowerAI Vision).

Go here to learn more about trial access (Scroll down to the Give it a try section).

This code pattern has been updated with screenshots and instructions for PowerAI Vision 1.1.3, but the instructions are the same for IBM Maximo Visual Insights.

Jupyter Notebooks

The code included in this code pattern runs in a Jupyter Notebook. The notebook itself does not require Maximo Visual Inspection or Power Systems (only access to the deployed API). To run the Jupyter Notebook locally, install it using Anaconda. The installation instructions are here.

Steps

  1. Create a dataset in Maximo Visual Inspection
  2. Train and deploy
  3. Automatic labeling
  4. Train and deploy
  5. Run the notebook
  6. Create the annotated video

Hint: If you need a shortcut, you can import the dataset from data/examples/dataset_auto_labeled.zip, train and deploy that dataset, and then run the notebook (but you'll get more out of this if you go through all the steps).

1. Create a dataset in Maximo Visual Inspection

To create a new dataset for object detection training from a video, use the Maximo Visual Inspection UI and start with a small manually annotated dataset (we'll expand on it with automatic labeling later).

2. Train and deploy

Train the model

Deploy the model

3. Automatic labeling

We use the first deployed model that you trained with manually annotated frames and use inference to automatically label more cars in your training video.

Frames are captured at the specified interval and labels are added by using the specified trained model. By default, the automatically added labels are light red. After processing, you can manually add labels to the frames that have been auto labeled and you can manipulate (move, resize) the labels that were automatically generated. If a frame with automatically generated labels is edited, all labels on the frame are converted to manual labels.

4. Train and deploy

Repeat the above train and deploy process with the newest dataset which was enhanced with automatic labeling.

This dataset has many more frames and labeled objects. It will create a much more accurate model.

5. Run the notebook

The code included in this code pattern runs in a Jupyter Notebook. After you configure the URL of your deployed model in the notebook, you can just run it, read it, and watch the results.

6. Create the annotated video

You can create an MP4 video from the annotated frames if you have a working installation of ffmpeg. The command is commented out as the last cell of the notebook. You can run it from there, or use the script in tools/create_video.sh. The script takes the output directory (with the annotated frames) as an argument like this:

cd powerai-counting-cars
./tools/create_video.sh notebooks/output

Note: There is also a tool to create a gif from the video. We used that to show the sample output below.

Sample output

As the notebook cells run, check for errors and watch the progress indicators. After the video has been annotated, the frames will play (like a video) in the notebook. The notebook playback is usually slow. If you used ffmpeg to create an annotated video, you can play it back at full speed.

Example annotated video: here

Example notebook with static output: here

Example compressed and converted to gif:

video-to-gif

Troubleshooting

Links

Learn more

License

This code pattern is licensed under the Apache License, Version 2. Separate third-party code objects invoked within this code pattern are licensed by their respective providers pursuant to their own separate licenses. Contributions are subject to the Developer Certificate of Origin, Version 1.1 and the Apache License, Version 2.