yannickrud / made-project

Repository of Yannick Rudolf for the Methods of Advanced Data Engineering course at FAU SS24
https://oss.cs.fau.de/teaching/specific/made/
Creative Commons Attribution 4.0 International
0 stars 0 forks source link

Exercise Badges

Evolving Weather Trends in Germany: Insights from Historical Data

This project analyzes data from the German Weather Service(DWD) in relation to the change in the number of weather phenomena over time. The data is accumulated annually from weather stations in Germany. This means that the analysis of the data will change after the next update.

The DWD data is, like this project, under the Creative Commons BY 4.0 "CC BY 4.0" license.

The pipeline extracts the data from the DWD's Open Data Portal. The metadata and the data of the individual weather stations are extracted and stored in the project.sqlite database.

Image of Data engineering pipeline

Start the pipeline

./project/pipeline.sh

The following weather phenomena are observed within the data sets:

The reports as wells as the jupyter notebook for the exploratory analysis for the project can be fould in the project folder of this repository.

The following image shows that the number of weather phenomena increases over time. For a more detailed analysis please have a look at the Final report Overall occurences of weather phenomena per year

Files

Data report
Final report