datahq / dataflows

DataFlows is a simple, intuitive lightweight framework for building data processing flows in python.
https://dataflows.org
MIT License
195 stars 40 forks source link

Allow using singer.io taps and targets as processors #16

Open rufuspollock opened 6 years ago

rufuspollock commented 6 years ago

This would allow us to leverage a great set of existing integrations and greatly expand list of processors.

Tasks

rufuspollock commented 6 years ago

@akariv any thoughts?

akariv commented 6 years ago

I agree this is a good use case.

OriHoch commented 5 years ago

example of loading data from singer tap -

https://gist.github.com/OriHoch/087bc2ab49cb354009c1a021cf1ea5e9

to support the schema we will need to convert JSON Schema to Table Schema (couldn't find a library that does it)

rufuspollock commented 5 years ago

I would say this is a major priority if it were easy as it would massively expand the list of processors ...