WKirejew / SmokeOut

Data pipeline for startup SmokeOut
1 stars 0 forks source link

SmokeOut

Incubated in 2022 start-up of a mechatronical device connected with mobile app for tobacco addicts, aiming at cesseting and monitoring problematic behaviours during their road to full recovery. Thanks to author of the idea Bartek Kubrak and general technnological engineer Konrad Sejud

Requirements:

Apache airflow:

Typical installation of apache-airflow

Providers: [installation by pip install (provider)]

apache-airflow-providers-microsoft-azure \ apache-airflow-providers-microsoft-mssql

Establishing approperiate connections:

Azure Blob Storage using wasb Hook \ Microsoft SQL at Azure

Drivers:

ODBC Driver for SQL Server

Python Libraries:

pandas \ fastapi \ uvicorn[standard] \ pydantic \ azure-identity \ azure-storage-blob \ mysqlclient (for testing in mysql) \ SQLAlchemy \ ipynb (to import files from notebooks)

Generating inputs:

Using the project:

For purpose of writing new data into SQL database permission, contact:

wojtasss99@gmail.com

For read only:

DataBase looks like this: db