The goal of the project is to detect heavy objects in the City of Amsterdam using the Panorama images.
git clone https://github.com/Computer-Vision-Team-Amsterdam/Detecting-Heavy-Objects.gitdet2 environment.To create the environment from the .yml file, run
conda env create -f det2.ymlTo activate the environment, run
conda activate det2To deactivate the environment, run
conda deactivateFor this project we use an instance segmentation framework named Detectron2. From the provided models zoo, we start from a pre-trained model which uses the Base-RCNN-FPN configuration. The model performs both instance segmentation and object detection on a given image.
To train the model on your local machine, run
python training.py --name &{MODEL_NAME} --device cpu${MODEL_NAME} is the name you give to your model; can be anything.
The output files are stored according to the configuration file.
By default, output files are stored in the outputs folder.
To train the model on GPU, run
python run_train_on_azure.py --name ${MODEL_NAME} --subset train
Latest model is registered on Azure.
The output files are stored on Azure in the outputs folder and contain the following:
A trained weights file on the class "container" can be found here: Google Drive.
The output files are also downloaded locally in outputs/TRAIN_${MODEL_NAME}_${MODEL_VERSION} folder.
Once you have a trained model, you can visualize the training loss curves, accuracies and other metrics by running:
python -m tensorboard.main --logdir $PWD/outputs/TRAIN_${MODEL_NAME}_${MODEL_VERSION}To perform inference on your local machine, run
python inference.py --name ${MODEL_NAME} --version ${MODEL_VERSION} --device cpu
The default version is the latest version.
Currently, you need to make sure you have the MODEL_NAME with MODEL_VERSION locally as well as the data folder.
To perform inference on the validation set on GPU, run
python run_inference_on_azure.py --name ${MODEL_NAME} --version ${VERSION} --subset val To run inference on a test dataset with no corresponding annotation file, use --subset test instead.
The output files are as follows:
The trained model is downloaded locally at azureml-models/${MODEL_NAME}/${MODEL_VERSION}/model_final.pth.
The output files are downloaded locally in outputs/INFER_${MODEL_NAME}_${MODEL_VERSION} folder.
To visualize some detected containers, run
python predictions.py --name ${MODEL_NAME} --version ${MODEL_VERSION} --device cpu
Firstly, make sure you have the the model weights with ${MODEL_NAME} and ${MODEL_VERSION} locally in
outputs/${MODEL_NAME}_${MODEL_VERSION} folder.
To detect containers on a given image, run
python predictions.py --name ${MODEL_NAME} --version ${MODEL_VERSION} --image ${PATH}
To get predictions for a batch of images, place them at data/test (don't rename the test folder), then run
python inference.py --name ${MODEL_NAME} --version ${MODEL_VERSION} --device cpu --subset test
Example:
python inference.py --name best --version 1 --device cpu --subset test
Output files are stored at outputs/INFER_${model_name}_${version}_${timestamp}
To compute and store evaluation metrics, run
python evaluate.pyWe use the following metrics:
We calculate AP and AR for the following list of maximum detections: 1, 10 and 100.
We calculate AP and AR for the following list of bounding box areas: small, medium and large , where
We evaluate both bounding boxes results and segmentation results.
Precision measures how many of the predictions that the model made were actually correct. For each bounding box, we measure the overlap between the predicted bounding box and the ground truth bounding box, i.e. intersection over union (IoU). Precision is calculated based on the IoU threshold.
Recall measures how well the model finds all the positives out of all the predictions it makes. For example, we can find 80% of the possible positive cases in the top K predictions. Recall is also calculated based on the IoU threshold.
In COCO, mAP=AP and mAR=AR.
AP is calculated by taking the area under the precison-recall curve. If we have multiple IoU thresholds, we average the AP corresponding to each threshold.
When we compute AR, we again take a set of IoU values [0.5, 0.55, 0.60, 0.65, 0.70, 0.75, 0.80, 0.85, 0.90, 0.95]. Then, for each category, compute the recall at detecting that category at the specific IOU threshold. Then consider two cases
Both cases can be considered over a varying amount of maximum detections, 1, 10, 100.
For 1, we compute the recall for each category, and then take the average of those values over all classes.
For 2, we average the recall at each of the IoU threshold, and then take the average of this average for each class. This is our AR@{D} for D detections.
The model detects construction containers in images that are provided by the panorama car. This car which drives through the city and takes panoramic images of the streets. Subsequently, these images are uploaded to the panorama API.
Given a day D when the panoramic car drove through the city, we can visualize both the trajectory of the car and the detected containers on this trajectory. To create the map, navigate to the visualizations folder and run
python -m visualisations.daily_trajectoryThe day to plot can be configured in the script. The HTML map is created in the same folder.
We first finish annotations Download images Merge all folders into one Download the annotation from AzureML
Run clustering notebook. Now we have 3 folders: train, validation, test
Now we need to resize all images to 2000x4000 aka run correct_faulty_panoramas()
This will generate folder with upscaled images. These images must be manually moved and replace the downscaled ones in the 3 folders that result from the notebook.
Then we apply the data format converter aka run The converter on the output of the correct_faulty_panoramas()
These output files we can rename and upload to Azure.
Triggered automatically by PR to development or main branch. Not triggered automatically in case of documentation or pipeline update.
Automatically triggered pipeline runs can use these parameters only in its default values. Intended only for manual runs for pipeline or build test reasons.
Triggered automatically by push to development or main branch. Not triggered automatically in case of documentation or pipeline update.
Automatically triggered pipeline runs can use these parameters only in its default values. Intended only for manual runs for pipeline or build test reasons.
Builds docker images and stores them in pipeline artifacts.
Additional info:
Deploys the docker images to DEV environment.
Prerequisites:
Additional info:
Deploys the docker images to TEST environment.
Prerequisites:
Additional info:
Deploys the docker images to ACC environment.
Prerequisites:
Additional info:
Tests vulnerabilites detected by Microsoft Defender for Cloud after image deployment to ACR.
Prerequisites:
Additional info:
Deploys the docker images to PROD environment.
Prerequisites:
Additional info: