search

sparsecode / DaFlow

Apache-Spark based Data Flow(ETL) Framework which supports multiple read, write destinations of different types and also support multiple categories of transformation rules.
Other
26 stars 13 forks source link

Add support for schema registry module with support for multiple versions of schema #27

Open abhioncbr opened 5 years ago

abhioncbr commented 5 years ago

Functionality / Module required for validation and transformation of a feed schema. Maintaining the versions of a schema is one of the basic requirement. Also, the schema should be easily accessible from various endpoints based on different methods.

Schema registry framework in future can be extensible for storing different vendors data-types mapping.

abhioncbr commented 5 years ago

Schema module availability through grpc & thrift initially will be good. Later we can expose through rest api too.