Closed s-minoo closed 2 years ago
Rephrasing this a bit, to make the link more explicit, and removing the data type parts because that could be a next step
Streaming (RDF) data can be accessed through various different interfaces (websockets, http, etc..), whilst current data processing frameworks (e.g. Apache Flink, NiFi, Spark, ...) have different configurations for the same interface. With a generic I/O interface that can be applied to different frameworks, we provide an abstraction layer that makes it easier to onboard existing data onto Solid pods. Furthermore, this would enable a strong foundation to a more modular data processing workflow architecture without reliance on existing tech stack on data processing.
This is a good technical challenge!
In the context of this repository, we try to be pin things to a specific use case, such that there can be a specific demonstrator. (See for instance #1.)
Are there use cases available that would be a good match, and can we apply them?
@bjdmeest @s-minoo What does "FFI" stand for?
@bjdmeest @s-minoo What does "FFI" stand for?
https://en.wikipedia.org/wiki/Foreign_function_interface , I updated the description
@s-minoo Did you have the chance to look into making the necessary changes?
This one has evolved into #47 where there's also a need of I/O alignment.
Does that mean that this one is no longer needed?
I think it's better to close this one. Since #51 will eventually be the end result of this challenge.
Pitch
Current data processing components have different configurations for the same I/O type (websockets, http, etc..). Furthermore, the representation of the data types is also different across the different processing components. A generic I/O interface and data types, independent from the different languages, would enable a strong foundation to a more modular data processing workflow architecture without reliance on existing tech stack on data processing.
Desired solution
Acceptance criteria