Add minimalist DHT API for scripting

[gavin-norman-sociomantic]

Currently, initialising and using a DHT client involves a lot of boilerplate, making it unsuitable for use in quick D scripts.

The ideal would be for things like this to work:

void main ( )
{
  Dht.connect("dht.nodes");
  Dht.update!(S)(key,
    ( ref S deserialized_record )
    {
      // changes made to deserialized_record will be written back to the DHT
    }
  );
}

[Geod24]

Wouldn't it be better to use dhtcli in D scripts ? That would also ensure it can be used in any script, not only D

[gavin-norman-sociomantic]

The basic problem is that we need an easy way to modify records in a DHT, for testing. (Read a record, deserialize the struct, change some fields, serialize, write back.) There's no easy way to do this with dhtcli. We did discuss using some kind of struct <-> json converter, say, but I think that'd be a lot more work to develop than the above suggestion would.

[Geod24]

Read a record, deserialize the struct, change some fields, serialize, write back.

What I did back in the day for a very similar use case was:

• Read the data from DHTCLI using swarm.util.RecordStream. DHTCLI was spawned via EpollProcess IIRC.
• Deserialize using the regular MultiVersionDecorator
• Serialize to JSON using Vibe.d's vibe.data.json : serializeToJsonString
• Output to stdout

That utility was less than 100 LoCs (but only handled deserialization). @christoph-wasicki-sociomantic might still have it hanging around.

[gavin-norman-sociomantic]

Right, the point is that it requires knowledge of the serialized struct, though, right? That alone locks us into the D domain.

[Geod24]

Right, the point is that it requires knowledge of the serialized struct, though, right? Right, the point is that it requires knowledge of the serialized struct, though, right?

Sorry, I don't understand your point here. You have a similar use case in profileview, where one can give either the channel or the struct type as argument and have the other one being automatically deduced. The only difference is generating machine readable output as opposed to human readable one. And once it's in JSON, it's accessible to anyone, not just D.

[gavin-norman-sociomantic]

My point is that it always requires D code that knows the type of the struct to be deserialized, either to implement a DHT client API that automatically deserializes or a struct -> json deserializer. I guess you weren't suggesting otherwise, though.

It's really a question of what is simpler to implement and more convenient to use.

• Simplified DHT API: pretty simple to implement; extremely high convenience for users (just set the fields you want to change).
• Struct <-> json serializer: pretty simple to implement (as you explained above); moderate convenience for users (have to manipulate json strings).

[gavin-norman-sociomantic]

I agree that a solution that works via json would be more generic, of course, but we have to think about what is more directly convenient to us right now. (We can also implement both solutions, down the line. There's no reason to have to choose one or the other.)