rheem-ecosystem / rheem

Rheem - a cross-platform data processing system
https://rheem-ecosystem.github.io
5 stars 0 forks source link

Introduce plug-ins #16

Closed luckyasser closed 7 years ago

luckyasser commented 7 years ago

From @sekruse on August 1, 2016 15:13

The optimization process of Rheem is currently affected by means of Platforms. Those can be registered with a RheemContext and all their ExecutionOperators etc. will then be considered when executing a RheemPlan. However, this abstraction is not really appropriate as there is a different between platforms and "plug-ins". A platform provides an Executor to have specific ExecutionOperators executed. However, neither the ExecutionOperators nor the mappings from Operators to ExecutionOperators nor the ChannelConversions are necessarily described (exhaustively) by a platform.

To this end, we should introduce plug-ins decouple execution platforms from the above mentioned components in Rheem. A plug-in provides Mappings, ChannelConversions, ExecutionOperators, as well as Configuration properties and requires Platforms. Also, each platform should ship with a "basic" plugin.

Copied from original issue: daqcri/rheem#9