search

pombreda / appscale

Automatically exported from code.google.com/p/appscale
0 stars 0 forks source link

Integrate MapReduce functionality #68

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
Work has been done to write MapReduce functions for a number of scientific
apps. Need to integrate this work into AppScale for the community to use.
Thus there are two necessary tasks:

1) Provide a library for users to call Map and Reduce in their GAE app.
2) Provide a method to expand on the Mappers and Reducers available.

Original issue reported on code.google.com by shattere...@gmail.com on 17 Jun 2009 at 10:32

GoogleCodeExporter commented 9 years ago 
Started. Need to provide a library that provides a simple mechanism to run the 
code
in parallel (over AppScale) or serial (for testing). 

For now, the computation will be done over existing machines. Future work may 
want to
spawn dedicated machines for this purpose ala Amazon's Elastic MapReduce.

Original comment by shattere...@gmail.com on 17 Jun 2009 at 11:19

GoogleCodeExporter commented 9 years ago 
Delayed for now. Many issues exist that need to be addressed before this can be
implemented and persistence is a higher priority feature that can be 
implemented faster.

Will return to this after persistence is implemented.

Original comment by shattere...@gmail.com on 22 Jun 2009 at 8:06

GoogleCodeExporter commented 9 years ago 
Done! Will be in next release. Currently supports mappers and reducers in the
following languages: Python, Ruby, Perl, Java. Other language support can be 
added
relatively easily for interpreted languages.

A sample mapreduce app will be included in the next release and the appropriate 
API
calls will be discussed on a wiki page to come soon.

Original comment by shattere...@gmail.com on 15 Sep 2009 at 6:59