HandleOrForward functionality.

[brewkode]

Any operation(get or put) on the cluster, should translate to a HandleOrForward operation. based on the key, the node should be able to localize if it can handle the request or it should be able to forward the request an appropriate node.

@ashwanthkumar below is my stab at at the contract. Thoughts? def handleOrForward(key: K): Node

PS: Is this a good way to write down issues.

[ashwanthkumar]

Contracts are a great way to write issues 👍

HandleOrForward would be more like request intercepter right? How would returning Node help?

[brewkode]

Can you share your thoughts, please?

[brewkode]

https://github.com/uber/ringpop-go/blob/dev/ringpop.go#L710

I visualized it as a way to know if it should be handled by this node or forwarded to someone else, both of which could be abstracted out via the Node interface. but, thinking about it, let's say, I've the Node, what would I do with it? How would I ask for data from it? It would translate to HandleOrForward to that node too, which wouldn't help.

So, HandleOrForward should return Either[V, Node]. V if there's value in this node or the actual Node to talk to get the data.

If we went with abstracting this out into a function, we would have a function wrapping this, to return just Option[V], which takes care of forwarding logic too.

[ashwanthkumar]

Given the fact that we're going to take care of forwarding (isn't?) - shouldn't HandleOfForward take in a handler which would be invoked with an object of T <: Request (either a GetRequest or PutRequest) ?

[ashwanthkumar]

Even better would be if we can differentiate GetRequest or PutRequest - we could just call in an implementation and call the corresponding def get[R <: Response](r: GetRequest): R or def put[R <: Response](r: PutRequest).

[brewkode]

Pl take a stab at putting down the contract so that it's clear.

[ashwanthkumar]

sealed trait Node {
  def get[R <: Response](r: GetRequest): R
  def put[R <: Response)(r: PutRequest): R
}

abstract class AbstractNode extends Node {
  def handleOrForward(key: Key, r: Request): Response = {
    if (shouldHandle(key)) {
      // Handle the request locally
    } else {
      forward(key)
    }
  }
  def shouldHandleKey(key: Key): Boolean = { ... }

  def forward(key: Key): Response = { ... }
}

What do you think?

[brewkode]

This looks ok. Something that's not coming out clearly is forward. forwarding is based on the strategy by which we do sharding or partitioning. So, we should have some way to get that strategy and zero-in based on that.

And, when you say, Node - you referring to a Shard of sorts, right?

[ashwanthkumar]

And, when you say, Node - you referring to a Shard of sorts, right?

I'm not sure. To me Node represents an instance of the process. And Shard is something that's handled by the Node. In our case Node would be serving multiple Shards.

This looks ok. Something that's not coming out clearly is forward. forwarding is based on the strategy by which we do sharding or partitioning. So, we should have some way to get that strategy and zero-in based on that.

I thought we discussed we'll only be supporting CH for the first cut. So I took that for granted, else we should have a PartioningScheme (as a trait or abstract class) and ask for an implementation for that. We can also start off with only a default implementation being CH.

[brewkode]

Node serving multiple Shard instances is perfect. Node would then need a way to know which shard the request needs to go to. Or, that would be delegated to the store internally?

I also think, having the partitioner come through explicitly via the contract is better, because, it would allow us to extend this whenever we need without having to worry about making too many changes. We know this is something we need to do anyways. Vinoth was already bringing up the prefix scan use-case :)

[ashwanthkumar]

I kinda visualise the Partitioner as something like this. Thoughts?

trait Partitioner {
   def shard(r: Request): Array[Byte]
   def find(key: Array[Byte], replicaCount: Int): List[NodeInfo]
   def find(key: Array[Byte]) = find(key, 1)
}

@vinothkr 😱 I know you would do something like this.

[ashwanthkumar]

Picking this up!