A light framework for creating data streams. Setting up streams through configuration in the Yaml file. There is a schedule, task pools, concurrency limitation. Works quickly, does not require a lot of resources. Runs on Windows and Linux. Flow run in parallel via threading library. Internally SQLite Database. Native data transformation. There is a web interface.
At the moment there are connectors to sources
Storages
It is highly recommended to install in a virtual environment.
Flowmaster needs a home, '{HOME}/FlowMaster' is the default,\ but you can lay foundation somewhere else if you prefer\ (optional)
For Windows
setx FLOWMASTER_HOME "{YOUR_PATH}"
For Linux
export FLOWMASTER_HOME={YOUR_PATH}
pip install flowmaster==0.7.1
# For install web UI.
pip install flowmaster[webui]==0.7.1
# Optional libraries.
pip install flowmaster[clickhouse,postgres,mysql,yandexdirect,yandexmetrika,criteo,googlesheets]==0.7.1
flowmaster run --help
flowmaster run
Pavel Maksimov
My contacts Telegram, Facebook
Удачи тебе, друг! Поставь звездочку ;)