Closed ghost closed 4 years ago
Looks really interesting. I'll have a look at Benthos & report back here with a set of possible paths to integration. Hope to get to this this week
Looked at Benthos, it's great! Not so sure we need to write an integration at this time tho
Benthos is a golang project can can transform any data from various sources, and i think a starlark wrapper would be hugely valuable.
https://github.com/Jeffail/benthos https://github.com/benthosdev/benthos-lab
vid: https://www.youtube.com/channel/UCjIYEhBrw3GQwpRWe1asufg/videos
This would make it easy to also watch the foreign data sources and the Benthos transformation can run and only pick up new data and pump it into qri as a new revision of the data. SO its all should be self running essentially.
Benthos is plugable also.