search

usdot-fhwa-stol / carma-streets

CARMA Streets is a component of CARMA ecosystem, which enables such a coordination among different transportation users. This component provides an interface for CDA participants to interact with the road infrastructure. Doxygen Source Code Documentation: https://usdot-fhwa-stol.github.io/documentation/carma-streets/
10 stars 10 forks source link

readme

CI Build Status Sonar Code Quality
CI Quality Gate Status

DockerHub Release Builds

Scheduling Service Message Services Intersection Model Signal Opt Service Tsc Service
Docker Cloud Build Status Docker Cloud Build Status Docker Cloud Build Status Docker Cloud Build Status Docker Cloud Build Status

DockerHub Release Candidate Builds

Scheduling Service Message Services Intersection Model Signal Opt Service Tsc Service Sensor Data Sharing Service
Docker Cloud Build Status Docker Cloud Build Status Docker Cloud Build Status Docker Cloud Build Status Docker Cloud Build Status Docker Cloud Build Status

DockerHub Develop Builds

Scheduling Service Message Services Intersection Model Signal Opt Service Tsc Service Sensor Data Sharing Service
Docker Cloud Build Status Docker Cloud Build Status Docker Cloud Build Status Docker Cloud Build Status Docker Cloud Build Status Docker Cloud Build Status

CARMA Streets Base Image Builds

Build and Push Base Images

CARMA Streets is a component of CARMA ecosystem, which enables such a coordination among different transportation users. This component provides an interface for CDA participants to interact with the road infrastructure. CARMA Streets is also an edge-computing unit that improves the efficiency and performance of the Transportation Systems Management and Operations (TSMO) strategies.

Architecture

CARMA Streets architecture is based on a scalable services and layered architecture pattern that allows for easy deployment. Service components are packaged to contain one or more modules (classes) that represent a specific reusable function (e.g., decode a particular ASN.1 message) or an independently deployable business function (e.g., control interface to a signal controller). Services interact with each other via lightweight messaging service (e.g., Kafka) which allows for them be deployed either together or distributed for scalability and performance. A high-level abstract view of the architecture to communicate the design pattern is shown in Upcoming Figure. A more detailed Unified Modeling Language class and packaging diagrams to define the interfaces between services and layers and their interactions will be developed and documented here during implementation following an Agile Development Methodology.

Deployment

Docker is the primary deployment mechanism to containerize one or more services. The CARMA Streets application and other major frameworks such as Kafka will run in their own separate containers. This document will be updated with a detailed Docker deployment strategy during later design phases.

Development

This repository includes configurations for devcontainer VSCode extension. This extension allows us to standup a containerized development environment. More information about the CARMA Streets Dev Container Setup can be found here.

Base Images

To make creating new CARMA Streets services easier and to make our CI/CD more efficient, we have introduced new CARMA Streets base images. These images can be used as a starting point and common build/runtime environment for CARMA Streets services. There are currently two CARMA Streets base images , for which documentation and Dockerfiles can be found Streets Service Base and Streets Service Base Lanelet Aware.

CARMAStreets

The primary carma-streets repository can be found here and is part of the USDOT FHWA STOL github organization. Documentation on how the carma-streets functions, how it will evolve over time, and how you can contribute can be found at the above links, as well as on the Doxygen Source Code Documentation.

Data Collection

CARMA Streets services have several source from which data about usecase performance can be pulled. Kafka Message Topics can be pulled using the collect_kafka_logs.py script and includes message traffic on a configurable list of topics. Service log files can be found in a given CARMA Streets service logs/ directory. The collect_service_logs.sh adds all log files in this directory to a zip file and then deletes the originals. Some CARMA Streets services can configurable log performance in .csv file in the logs/ directory.

Collect Kafka Logs

This script uses docker exec to ssh into a running kafka container. Then using kafka container scripts to read all kafka data from a list of provided topics. 

usage: collect_kafka_logs.py [-h] [--start_timestamp START_TIMESTAMP] [--end_timestamp END_TIMESTAMP] [--start_hours_ago START_HOURS_AGO] [--end_hours_ago END_HOURS_AGO]
                             [--topics TOPICS [TOPICS ...]] [--timeout TIMEOUT] [--zip ZIP]
                             outdir

Script to grab data from kafka

positional arguments:
  outdir                Folder name for the resulting folder logs are placed in

options:
  -h, --help            show this help message and exit
  --start_timestamp START_TIMESTAMP
                        Unix timestamp (seconds) for the first message to grab. Exclusive with start_hours_ago.
  --end_timestamp END_TIMESTAMP
                        Unix timestamp (seconds) for the last message to grab. Exclusive with end_hours_ago.
  --start_hours_ago START_HOURS_AGO
                        float hours before current time to grab first message. Exclusive with start_timestamp.
  --end_hours_ago END_HOURS_AGO
                        float hours before current time to grab last message. Exclusive with start_timestamp.
  --topics TOPICS [TOPICS ...]
                        list of topics to grab data from
  --timeout TIMEOUT     timeout for receiving messages on a topic, default is 5 seconds
  --zip ZIP             bool flag. When set to true, folder is compressed into a zip file.

Collection Service Logs

This script collects all CARMA Streets service log files, adds them to a zip file.

usage: collect_service_logs.sh [-h | --help] [--output <zip_file_name>.zip]
                                     [-c | --clear]

options:
-h, --help                Show usage
--output, -o              Name of the resulting zip file container CARMA Streets Service logs
--clear, -c               Adding this flag will delete log directories after creating zip.

Contribution

Welcome to the CARMA contributing guide. Please read this guide to learn about our development process, how to propose pull requests and improvements, and how to build and test your changes to this project. CARMA Contributing Guide

Code of Conduct

Please read our CARMA Code of Conduct which outlines our expectations for participants within the CARMA community, as well as steps to reporting unacceptable behavior. We are committed to providing a welcoming and inspiring community for all and expect our code of conduct to be honored. Anyone who violates this code of conduct may be banned from the community.

Attribution

The development team would like to acknowledge the people who have made direct contributions to the design and code in this repository. CARMA Attribution

License

By contributing to the Federal Highway Administration (FHWA) Connected Automated Research Mobility Applications (CARMA), you agree that your contributions will be licensed under its Apache License 2.0 license. CARMA License

Contact

Please click on the CARMA logo below to visit the Federal Highway Adminstration(FHWA) CARMA website. For technical support from the CARMA team, please contact the CARMA help desk at CAVSupportServices@dot.gov.

CARMA Image