usdot-fhwa-stol / carma-analytics-fotda

CARMA Analytics is a secure, scalable and cloud agnostic data architecture to enable research analysis on big data sets by defining repeatable and scalable processes for ingesting data, creation of a data repository to fuse, process and perform quality assurance.

Apache License 2.0

3 stars 3 forks source link

Add MAP and SDSM parsing #23

Closed paulbourelly999 closed 8 months ago

paulbourelly999 commented 8 months ago

PR Details

Description

Add methods to parse SDSM and MAP data from Kafka topic logs Added logic to account for multiline json

Related GitHub Issue

Related Jira Key

CDAR-535

Motivation and Context

Data Analysis (Message Frequency Plots)

How Has This Been Tested?

Types of changes

[ ] Defect fix (non-breaking change that fixes an issue)
[x] New feature (non-breaking change that adds functionality)
[ ] Breaking change (fix or feature that cause existing functionality to change)

Checklist:

[ ] I have added any new packages to the sonar-scanner.properties file
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[x] I have read the CONTRIBUTING document.
[ ] I have added tests to cover my changes.
[ ] All new and existing tests passed.