search

tabaddor / av-swe-guide

An autonomous vehicle guide for computer science students and software engineers
MIT License
57 stars 7 forks source link
autonomous-vehicles computer-science guide industry roadmap self-driving-car software software-development software-engineering study-plan

readme

Software Engineering for Self-Driving Cars Guide :car:

Welcome to the self-driving cars guide for computer science students, software engineers, and those looking to develop skills in autonomous vehicle software.

An autonomous car is a vehicle capable of sensing its environment and operating without human involvement. A human passenger is not required to take control of the vehicle at any time, nor is a human passenger required to be present in the vehicle at all. An autonomous car can go anywhere a traditional car goes and do everything that an experienced human driver does. source

Table of Contents

Introduction

How to Contribute

:white_check_mark: This is an open source guide, so contributions are very much welcome. To contribute:

Fork this repository and clone the repository

git clone https://github.com/YOUR-USERNAME/av-guide

Change into the repository's directory

cd av-guide/

Make the changes on your forked version, and then submit a pull request which will be reviewed shortly after.

Contribution Guidelines

View the complete contribution guidelines, here.

Getting Started w/ the Guide :tada:

Introduction

You can use this guide however you like. If you prefer to follow along material by material, go ahead. If you want to skip ahead, come back, skip ahead again, do as you wish. To get started with basic navigation and resources, briefly skim through the below tips.

Navigating

Navigating this project should be fairly straightforward and simple. But, to help you along any struggles along the way, here is how to navigate to some specific things:

README Document

This README document is the central documenation for this guide. At the top, is the Table of Contents which you can use as a main directory.

Contributing Document

The contributing document provides details regarding contributing guidelines, issues, feature requests, and the repository vision. Before submitting an issue or pull request, please briefly review it.

Sections

The table of contents of this guide is here in this README document. Feel free to jump around from section to section, or skim through a section's resource. These sections will be continuously updating and improving with new material and resources.

Community :speech_balloon:

As this community grows, @tabaddor will look into creating a community Slack and/or Discord. For the time being, all communication will be exclusively on GitHub.

Pre-Requisites (and resources)

Some basic knowledge may be assumed, but below are important topics to review.

Safety is Critical :vertical_traffic_light:

Self-driving cars are safety critical. A safety critical system can be defined as something that:

comprises everything (hardware, software, and human aspects) needed to perform one or more safety functions, in which failure would cause a significant increase in the safety risk for the people or environment involved.

It is easy to acknowledge that autonomous vehicles need to be safe, but as a developer or someone working on self-driving systems, this is incredibly important to keep in mind. Working with safety critical systems differs greatly from working on a majority of the other systems.

IOS 26262 Safety Standard

To mitigate risk for developers, teams and companies developing safety critical systems typically have to follow safety standards, such as ISO 26262. ISO 26262 is one of the more well-known safety standards that you should be aware of regarding autonomous vehicles.

I recommend briefly skimming through ISO 26262 to get an idea of the standard, but to summarize some key points:

Standards aid in making sure the code you write as a developer is carefully evaluated, inspected by others, and tested during the development process to mitigate risk. This is an essential component of software engineering for any safety critical system, as a lot of emphasis is placed on "careful coding."

Example of a Safety Critical System for Self-Driving Cars

Examples of ADAS include adaptive cruise control and pedestrian crash prevention.

The gist? Always remember that safety is critical. If you are interested in learning more about software engineering processes for safety critical systems, I highly recommend you take a look at Phil Koopman's page (linked in the Resources section).

High Impact Work :arrow_up_small:

Software engineers working on self driving cars will:

1. Introduction

This introductory section will provide high level overviews about the history of autonomous vehicles, basic autonomous vehicle taxonomy/terminology, and the current state of self-driving cars. This section is not overly technical, but interesting and important nonetheless.

Below is a typical software stack for a self-driving car. It should be pretty self-explanatory, however, don't get bogged down in the details, it will all fit together at the end (taken from Hyundai-Aptiv Webinar).

hyundai-aptiv-webinar-softwarestack

History of Autonomous Vehicles :scroll:

Taxonomy

Levels of Autonomy :oncoming_automobile:

levelsofautonomy

Summary:

Section Terminoloy:

2. Software and Hardware

As a software engineer working on self driving car technology, it is important you have an understanding of not only the software, but the hardware involved. You don't need to be an expert in computer architecture or circuits, but a general understanding and appreciation of hardware is important. With that said, let's dive into the software and hardware architecture of self-driving cars.

Sensors and Hardware Architecture :battery:

Hardware in the Loop (HiL)

Hardware in the Loop is a technique that is used in the development and testing of real-time embedded systems. Working as a software or computer engineer at a self-driving car compnay, you may work on the Hardware in the loop team.

Below is an image of the various sensors in Google's self driving car. Sensors are devices which mesaure some sort of physical property and records or responds to the data.

Sensors work together to provide the car with visuals of its surroundings to ultimately help detect pedestrians, other cars, etc and feeds the data into learning algorithms that aid with motion planning and path prediction. You'll work with the actual computer vision algorithms that deal with the sensor data and images in the Perception section, but for now, just understand how this piece of hardware ties into the self-driving vehicle as a whole. Read HERE for more information about how a self-driving car uses sensors to see.

Software Architecture

The high level idea behind self-driving car software architecture and stacks is separating the main vehicle functionalities into modules (perception, planning, and controls), and allowing them to work together/communciate.

autoware-selfdrivingunit-stack

Section Tutorial:

Section Terminology:

3. Developing for Safety :traffic_light:

Like stressed earlier in the motivation section, SAFETY IS CRITICAL. As a software engineer or developer working on autonomous, safety critical systems, it's especially important your code applies to safety standards, is tested rigorously (manually and automatically), and more which will be covered in this section.

Safety Assurance for Self-Driving Cars

Unfortunately, sometimes things go wrong. That is the challenege.

Software Engineering Activites:computer:

As a software engineer, only around 10% to 30% of your time will be spent actually writing new code. The remaining time will be spent reading, understanding, debugging, testing, and communicating code. That is why, especially for a safety critical system like a self-driving car, you have a solid understanding of software engineering activities. We will focus on the big three that are: Testing, Code Quality and Metrics, and Productivity.

Testing

Testing is expensive. But it is also crucial for self-driving cars. Testing the vehicle modules as a whole may take place in simulations, but you will still need the basics of software development testing down.

You'll hear the terms validation and verification, and be aware that though the sound similar, they are widely different terms. Validation involves making sure your system meets requirements. Verification invovles making sure your software system is correct. Validation and verification can go hand in hand.

Static Analysis- Debugging that is done by examining the code without executing the program

Dynamic Analysis- Testing and evaluation by executing the data in real-time

Code Quality/Metrics

What is defined as quality code and how quality code is measured will differ from organization to organization, and maybe even from team to team. Is your goal bug-free code? Is your goal minimal debugging time? Is your goal less lines of code? These are all things managers and teams consider, and important for you as a developer to keep in mind when working on safety critical systems like self-driving cars.

Productivity/Teamwork

For the most part, you won't be working on a system alone. You'll be on a team, and a lot of times, coding with another developer in what is called pair programming. Pair programming involves two developers working on a feature, issue, etc, however, they work on their own parts, and then convene later on to discuss and communicate.

Dr. Phil Koopman- a gold mine of autonomous vehicle safety research and information:

His YouTube channel can be found in the Resources section. Here are some noteworthy vehicles I recommend:

Section Terminology:

4. Deep Learning for Self-Driving Cars 🧠

The high level idea behind Deep Learning is building machine learning algorithms and functions that mimic the human brain in processing data for tasks including object detection, speech recognition, and decision making. If you haven't already, I highly recommend you check out Stanford's Machine Learning Course for free.

ML, DL, RL Review

Machine Learning

A specifc class of Neural Networks you should familiarize yourself with are Convolutional Neural Networks, which are involved in tasks including object detection, image processing, classification, and image segmentation.

Convolutional Neural Networks

Section Terminology

5. State Estimation and Localization

State estimation is the problem of determining the current state of a system, such as a vehicle. Localization is the implementation of algorithms for state estimation with an error of less than a small, non-zero value.

Least Squares

State Estimation and Kalman Filters

Pose Estimation

Section Terminology

6. Perception 👀

For the purposes of this guide, perception and computer vision are synonymous. Perception deals with training our vehicle to perceive its environment in order to act on what it perceives.

Computer Vision

Today, there are two main ways of performing vision: Classical vision and machine learning. This guide will provide resources for understanding and working with both methods.

Classical Vision
Vision with Deep Learning

Object Detection

Semantic Segmentation

Section Terminology

7. Motion Planning

The Problem

Motion planning, also known as path planning, is the the problem of finding a sequence of valid motions, configurations, steps, etc that allow an agent to go from its source location to goal destination. Motional planning has several key applications in robotics and autonomous vehicles, rightfully so. In this section, we will break down some key aspects of this function.

Planning

Mapping

Dynamic Objects

Section Tutorials/Projects

Section Terminology

8. Testing

Importance of Testing

Unit Testing

Integration Testing

Static and Dynamic Analysis

Simulation

Resources

AV Companies :briefcase:

A list of companies working on autonomous vehicle technologies. The purpose of this section is to include any relevant information about the companies and any open internship and full-time positions. Don't see your company, position, or a link is broken? Submit an issue or pull request!

Waymo

Waymo is an autonomous driving technology company that is a subsiduary of Alphabet, Inc.

Open Roles

Cruise

Cruise is a self-driving car company owned by General Motors (GM).

Open Roles

Ford

Ford Motor Company is an American automaker now building hybrid and electric vehicles, AVs, SUVs, and Crossovers.

Open Roles

Zoox

Zoox is an American Autonomous Vehicle company now a subsiduary of Amazon.

Open Roles

Tesla

Tesla is an electric car and solar/clean energy company.

Open Roles

Aptiv

Aptiv is an autoparts company shaping the future of mobility through green, connected ADAS and AV technology.

Open Roles

Rivian

Rivian is an electric vehicle manufacturer also working on AV technology.

Open Roles

HARMAN International

HARMAN International is a Samsung subsiduary focusing on car lifesyle, audio, and connected products, including ADAS.

Open Roles

FAQ :speech_balloon:

Q: Who can use this guide?

A: This guide is mainly geared towards computer science students and software engineers looking to learn more about the self-driving car industry. With that said, anyone is welcome to use on the guide, just make sure you have an understanding of the pre-requisites.

Q: How do I get started with this guide and roadmap?

A: Get started here! Official guide content starts here.

Q: Can I download this guide?

A: You may clone and work with this project on your local machine.

Q: I have a suggestion, how can I contribute to this guide?

A: Contribitions and suggestions are always welcome. First, read through this contributing document, then submit an issue and/or pull request.

Q: I don't know much about computer science/software, how can I still use this?

A: Some resources in this guide assumes a certain level of knwoledge of computer science and software engineering concepts.