search

mberneaud / us-weather-alerts

Le Wagon Data Engineering bootcamp final project
2 stars 1 forks source link

Create Python functions to scrape API and transfer data to data lake bronze layer. #6

Open mberneaud opened 6 months ago

mberneaud commented 6 months ago

Goal Create logic that allows us to scrape both historical data as well as currently active alerts from the National Weather Service API and store them in our data lake.

The pythonic API implementation in the NWSApy package can prove helpful in making our work easier.

The destination of the data is dependent on Janine's findings and will either be BigQuery or Firestore.

mberneaud commented 6 months ago

I am going to start with querying the data and storing them locally and then build additonal functionality to upload to GCP