search

snuspl / cruise

Cruise: A Distributed Machine Learning Framework with Automatic System Configuration
Apache License 2.0
26 stars 2 forks source link

Relation with Dolphin KeyValueStore #6

Closed jsjason closed 8 years ago

jsjason commented 9 years ago

From @jsjason on May 8, 2015 (original issue: cmssnu/elastic-memory#4):

@kijungshin, @dafrista, @jsryu21, and I had a discussion on the interface of Dolphin's KeyValueStore and EM. The first thought was to consider the two as components on the same layer with similar roles. In other words, they could have the same interface. However, there is a big difference: KeyValueStore does not consider a distributed environment, where EM does (the term KeyValueStore, in fact, does not refer to the common key-value store such as Memcached. It is simply a Service that delivers data between Tasks in a key vs. value form.). Instead, EM should have several KeyValueStores: one that stores local-only data, one that stores splittable data, and one that stores copied (broadcast) data. For now, we will rename the KeyValueStore to show that it is an EM replicate with no EM functions.

jsjason commented 9 years ago

From @bgchun on May 9, 2015:

Renaming KeyValueStore sounds fine. The interface name can also be agnostic to EM. The interface should be general enough for us to answer how it is easy to fit an existing ML framework.

Also, what we want is that we should be able to distribute a ML version without EM.

wynot12 commented 8 years ago

I'm closing it, because it's already resolved. The current EM uses MemoryStore, which is totally independent from KeyValueStore.