pi314ever / mathworks-physical-sensor-model

XPlore project for MathWork's physical sensor modeling.
2 stars 0 forks source link

Applying Machine Learning for the Development of Physical Sensor Models in Game Engine Environment

Contributors: Daniel Huang, Felix Meng, and Jason Zhu

Abstract

This project aims to generate a model which can calculate distortion effects efficiently and accurately given distortion parameters and an undistorted image. There are use cases such as full-simulation training for self-driving cars, where distortion model of the camera needs to be accurately and efficiently calculated from a simulated environment. Current un-distortion algorithms utilize standard iterative methods and are not optimized for real-time applications. We present two approaches to this problem: a GAN approach (led by Felix Meng) and a function approximation method (led by Daniel Huang). The GAN method showed promising results, but needed to be trained specifically per distortion parameter and cannot be generalized. The function approximation method was less visually accurate due to the point-wise nature of the model. However, it is able to generalize to different distortion parameters and is much more efficient than the GAN method.

Setup

Prerequisites

Steps

  1. Install poetry packages with poetry install
  2. Test environment setup with poetry run pytest

Dataset and Data Pipeline

Point map dataset

Dataset is not included by default (git ignored). To generate the dataset, run python src/data/generate_point_maps.py. This will generate the dataset in data/point_maps/ directory.

Useful Links

Other links