search

NICTA / scoobi

A Scala productivity framework for Hadoop.
http://nicta.github.com/scoobi/
482 stars 97 forks source link

Integrate with algebird #188

Closed avibryant closed 11 years ago

avibryant commented 11 years ago

There's lots of useful aggregations in https://github.com/twitter/algebird. They were originally developed for Scalding but there's no reason Scoobi couldn't steal them.

etorreborre commented 11 years ago

Hi Avi,

We will have definitely a look at Algebird. It is likely that we will evolve Scalaz if necessary based on similar concepts since @tonymorris is now part of the team, but we/I mostly need to understand what "integration" means. Does it mean that some Algebird classes are part of the Scalding API, offering some higher-level functionality? I need to re-read your team's post on the subject...

johnynek commented 11 years ago

The main interesting cases are the aggregation monoids (CountMinSketch, BloomFilter, HyperLogLog). Doesn't really seem like Scalaz is interested in that sort of thing now, but maybe they will get interested.

etorreborre commented 11 years ago

We're closing this suggestion while the API is still moving. We will probably open specific issues on the subject in the future.