I just recently found DDF and found it to be very appealing to do data analytics, due to its promise of programming expressiveness (R-style), scalability, .... I am still very new with DDF, and I want to ask a question here, as i can't find suitable forum or other discussion channels to ask this kind of question.
This is my question: Are there other backends built for DDF (either in production, experiment, coding, or planning stage)? At the outset DDF seems to be used on top of Spark natively. I feel that Spark is very heavy to begin with, though, especially if we are just starting some analytics. I think it could also use some lighter-weight frameworks, including Pandas or R itself as the backend. Then later on one can move on to Spark if necessity dictates (i.e. data has outgrown the backend). In this way, the analytics code are not changed.
Hi,
I just recently found DDF and found it to be very appealing to do data analytics, due to its promise of programming expressiveness (R-style), scalability, .... I am still very new with DDF, and I want to ask a question here, as i can't find suitable forum or other discussion channels to ask this kind of question.
This is my question: Are there other backends built for DDF (either in production, experiment, coding, or planning stage)? At the outset DDF seems to be used on top of Spark natively. I feel that Spark is very heavy to begin with, though, especially if we are just starting some analytics. I think it could also use some lighter-weight frameworks, including Pandas or R itself as the backend. Then later on one can move on to Spark if necessity dictates (i.e. data has outgrown the backend). In this way, the analytics code are not changed.
Wirawan