search

nathanaschbacher / nodiak

Nodiak is a Node.js client for the Riak Distributed Database
MIT License
43 stars 16 forks source link

Needs link walking support #3

Closed nathanaschbacher closed 11 years ago

nathanaschbacher commented 12 years ago

I need to come up with a useful scheme for handling link walking, something in the core client on the _object namespace that can be expanded in RObject or Bucket. Either an explicit walk method on RObject directly or a chain on the Bucket.object.get method perhaps?

nponiros commented 11 years ago

Hi Nathan,

has there any progress been made on the links front? I see me needing them for a project I work on and it would be nice if I could continue to use nodiak.

cheers

nathanaschbacher commented 11 years ago

Howdy,

I'm headed to bed, but I'll have an answer for you in the morning.

Cheers! -Nathan

On Aug 13, 2013, at 23:22, Nikolas Poniros notifications@github.com wrote:

Hi Nathan,

has there any progress been made on the links front? I see me needing them for a project I work on and it would be nice if I could continue to use nodiak.

cheers

— Reply to this email directly or view it on GitHub.

nathanaschbacher commented 11 years ago

So, Links.

You caught me at just the right time, because I'm adding 2i pagination and counters support as we speak.

Here's the issue with Link-Walking. It devolves into a special case of Map/Reduce, and thus contains all the caveats therein. So my suggestion to you would be, "Don't use Links." Not because I'm trying to be lazy, but because you can implement the same functionality on top of KV using regular asynchronous requests to Riak.

I hear you saying, "But that's extra work for me!" Yes, it is. Sort of. It's extra work now, but it's a lot less work than having to re-architect around KV down the road when you find out the hard way that Links aren't working for you anymore. Better still, it's not much extra work, and the story for your application and for Riak will be much happier if you don't use Links. You won't be hammering a coordinating node trying to service your query which will require aggregating potentially large result sets at-once.

When I can finally get a release version of chinood out the door (hopefully in the next few weeks, the graph fetch is the only part I don't have done yet) then I'll have a library that provides some sugar for doing those kinds of relationship fetches in-bulk and asynchronously for you by traversing the object graph similar to how Links work, but without the Links part :-) So it's both faster and plays nicer with the Riak cluster.

tl;dr, Don't use Links unless you absolutely have to, and if you think you absolutely have to, then I'd suggest you probably still don't need too, and further assessing how to model your data may be in order. In which case hit my fellow Bashoites on the mailing list or IRC, or keep pinging me since data architecture review and prototyping is like 1/3 of my job :-)

nponiros commented 11 years ago

Hi Nathan,

Thanks for your detailed reply. The reason I wanted to use links was just to have some data A point to some other data B with the help of riak. I guess my understanding of riak links was a bit too naive. I can actually easily store the key of B into the data of A and use that to access B when I need it. From what I understand this is the way you would recommend.

Great that you are adding support for 2i pagination. It is something I will probably need along the way ;) chinood also looks rather interesting. From a quick view it reminds me of mongoose (at least the model parts of it)

cheers

nathanaschbacher commented 11 years ago

You are correct sir. The better way to do it would be to store it on the object or in your own metadata and handle the fetching yourself asynchronously.

In fact, since that's my stated position I think I'm going to close this Issue. :-)