lisad / phaser

library for batch-oriented complex data integration pipelines
MIT License
3 stars 1 forks source link

Make pipelines able to run and load data and check errors from dynamic context #150

Open lisad opened 1 week ago

lisad commented 1 week ago

I started to write documentation for install.md, and found a couple things I really wanted to do for an example, but the library can't do yet.

from phaser import Pipeline, Phase, row_step

@row_step
def my_step(row):
    row['pudding'] = 'proof'

my_pipeline = Pipeline(
    phases = [
        Phase(steps=[my_step])
    ]
)

def my_import_data_view(request):
    data = request.body
    my_pipeline.load(data)
    my_pipeline.run()
    if my_pipeline.context.has_errors():
        return HttpResponse(status=401)
    else:
        return HttpResponse(status=200)