patterns-app / patterns-devkit

Data pipelines from re-usable components
BSD 3-Clause "New" or "Revised" License
106 stars 5 forks source link
data-analysis data-engineering data-pipeline data-pipelines data-science etl etl-framework etl-pipeline etl-pipelines functional-reactive-programming immutability pipelines sql

 

 

Patterns - Build data systems from re-usable sql and python components


Installation

pip install patterns-devkit

Usage

patterns create graph mygraph

This will create an empty patterns graph:

mygraph/
  graph.yml

Create a new python node:

cd mygraph
patterns create node mynode.py
mygraph/
  graph.yml
  mynode.py

Upload

To deploy a graph, you must sign up for a patterns.app account and login to authenticate the cli:

patterns login

Then you can upload your graph:

patterns upload

Other commands

You can see the full list of available cli commands:

patterns --help