search

mseminara0415 / Satellite-Tracker

ELT Data pipeline that uses a serverless event driven architecture to extract, load, and transform satellite tracking data provided from a couple different APIs, as well as scraped data from Wikipedia.
0 stars 1 forks source link

Add Json Validation for ISS Data Ingestion #4

Closed mseminara0415 closed 2 years ago

mseminara0415 commented 2 years ago

What's The Problem?

Our data is not currently validated and therefore our datalake is at risk of having invalid data.

Reference Material

Example data received from API:

{
  "name": "iss",
  "id": 25544,
  "latitude": -27.077564440134,
  "longitude": 77.403881493074,
  "altitude": 264.07695639739,
  "velocity": 17126.476285525,
  "visibility": "eclipsed",
  "footprint": 2816.5304217721,
  "timestamp": 1655077198,
  "daynum": 2459743.486088,
  "solar_lat": 23.195594381128,
  "solar_lon": 185.00692675203,
  "units": "miles"
}

Potential Solution

mseminara0415 commented 2 years ago

Closed with #5