Merged RJ's CORS fix

[chrisekelley]

I merged @rjsteinert 's PR enabling CORS support from here into my fork: pfrazee/dat-gateway#7

[RangerMauve]

Looking good! Lemme know when you're ready for the merge. :D

[chrisekelley]

@RangerMauve Super! can you please merge this? Let's get this unit committed, and then we'll start work on supporting base32-encoded subdomains.

[RangerMauve]

Merged this into the master branch so further PRs can go there.

[chrisekelley]

Yeah, good idea about the unit tests. Will do.

[RangerMauve]

I think that the test could just open a socket and succeed if it gets a non-zero number of peers.

[chrisekelley]

One another note - is there other system-level data we should get about the dats? If so, we may wish to create a url namespace such as /dat and branch off that. It would emulate how the dat API currently works

• /dat/network/connections/length - what I've currently named "/peers" - corresponds to dat.network.connections.length
• /dat/network/connections/- does this return a list of peers? Is that useful?
• /dat/doctor - dat doctor - diagnosing network quality
• /dat/log - view the history and metadata information for a dat.

[RangerMauve]

I'm not sure about the others, but /dat/log can already be done by looking at the hyperdrive after replicating it.

I think the others should be implemented when there's a concrete use-case for the application so save us work in the short term. :D

[chrisekelley]

Yes, certainly - and thanks for the tip on /dat/log. I may go ahead and create the /dat url and populate it with /dat/network/connections/length and /dat/log, but I mainly wanted to think over the namespace a little, just to make it a little more flexible.

[RangerMauve]

Were you thinking of HTTP endpoints? If it's websocket, perhaps you could use some sort of RPC over the websocket to reduce the amount of connections you make for stuff like this.

[RangerMauve]

Hyperdrive uses a protobuf based protocol over arbitrary streams. You could work off of what they did to have other types of messages.

[chrisekelley]

Wow - yeah, I'm really off-base with the whole http endpoint concept... This is my first dance at the websockets party... I did a little more reading on websockets today. Here's my next stab at it:

It probably makes sense just for the client to make a single connection to ws://localhost:3000, and then in the message tell the server what you want to do (/dat/network/connections/length, /dat/log, etc...) - in a stringified JSON object.

So, here is code pulled from the websocket-stream lib client example and modified slightly:

  var msg = { 
'type':'command',
'command':'/dat/network/connections/length'
}

  var stream = ws('ws://localhost:3000')
  stream.write(Buffer.from(JSON.stringify(msg)))
  stream.on('data', function(o) {
    var str = String.fromCharCode.apply(null, rawMsg)
        let msgArray = str.split(':')
        let count = msgArray[msgArray.length - 1]
       // do something with this data
    stream.destroy()
    t.end()
  })

For urls that you'd fetch, the json obj would look like this:

  var msg = { 
'type':'dat',
'url':'dat://bunsen.hashbase.io'
}

So, we'd have to refactor getWebsocketHandler a tad to adjust to this message concept. Instead of processingreq.url.split('/') for the urlParts, we'd wait for a message like this:

    ws.on('message', function (data) {
      if (!Buffer.isBuffer(data)) {
        ws.send(Buffer.from('fail'))
      } else {
var msg = JSON.parse(data)
//check the type
// do stuff based on the type.
        ws.send(Buffer.from('success'))
      }
    })

Does this idea work?

[RangerMauve]

Consider looking at prior art using RPC over websockets. I'd strongly advise using protocol buffers instead of JSON as it will be a lot more efficient in the long run.