How to Replicate in SDK?

[DougAnderson444]

dat-js and dat-node just kinda replicated out of the box. Without any createServer or fancy stream stuff.

Now that we're back to straight up archive = new Hyperdrive do we have to do the whole archive.replicate() business?

A) Is this implemented in SDK yet? and; B) If so, could we get a little paste love with an example? I see the archive.url but I'm struggling to get it to replicate onto another machine.

@RangerMauve

[RangerMauve]

The SDK handles replication and storage under the hood, so there's no need to call replicate. :D

Do any of the examples in the SDK's README work for you?

Are you using the web or node.js versions? The web version can have a hard time loading archives on people's home networks still. I'm hoping that hyperswarm will help with that.

Could you post the code you've tried so far? :O

[DougAnderson444]

Do any of the examples in the SDK's README work for you?

Yes, they work! (Great job, btw). I do get the dat url showing in the console. BUT, when I go to beaker or datbase.org, to try to pull the url up, it fails to show me anything. Hence my replicate question.

Are you using the web or node.js versions? The web version can have a hard time loading archives on people's home networks still. I'm hoping that hyperswarm will help with that.

Both? I am using the web version piped through browserify on Glitch.

Could you post the code you've tried so far? :O

Of course! I'm a big fan of Glitch.com, so I can paste my project here. https://glitch.com/~dat-sdk-humble-beginnings

// in public/client.js

const SDK = require('dat-sdk')
const { Hypercore, Hyperdrive, resolveName, deleteStorage, destroy } = SDK()

const filename = '/example.txt'

//  hyperdrive(storage, [key], [options])
const archive = Hyperdrive(null, {
  // This archive will disappear after the process exits
  // This is here so that running the example doesn't clog up your history
  persist: true
})

archive.ready(() => {
  const url = `dat://${archive.key.toString('hex')}`

  // TODO: Save this for later!
  console.log(`Here's your URL: ${url}`)

  // Check out the hyperdrive docs for what you can do with it
  // https://www.npmjs.com/package/hyperdrive#api
  archive.writeFile(filename, `Hello World on ${(new Date().toLocaleString())}`, () => {
    console.log('Written example file!')

    archive.readFile(filename, 'utf8', (err, data) => {
      if (err) throw err
      console.log(`Example text read is: ${data}`)
    })
  })
})

// now go to Beaker and pull up the URL...

[RangerMauve]

Yeah, the issue is that the web version of Dat has trouble connecting to home network. This is the same with Dat-js, too.

Basically, the way it works is it connects to a proxy server that does peer discovery and tries to make connections. When you try to load the site in Beaker, you start advertising your IP address on the P2P network. The gateway will discover it and attempt to connect to you. Since you're behind a NAT it'll usually fail.

There's two things you can do in this scenario: Either temporarily open it on a public gateway or add it to hashbase or a dat-store.

I'm currently working on hyperswarm-proxy which should help with getting through NATs.

I'll see what if there's a quick fix I can figure out though. 🤔

[DougAnderson444]

Yeah! Remember with Dat-Js we assigned a gateway option? Is there an equivalent with the SDK?

const Dat = window.datJs
        const dat = new Dat({
          persist: true,           // This flag will make it save the content locally to improve load speeds between refreshes
          gateway:  gateway
        })

[RangerMauve]

It's a little different now. The gateway before was for dat-gateway, but the new one is for discovery-swarm-web servers.

You can specify it in the constructor.

const sdk = SDK({
  swarmOpts: {
    discovery: 'whatever server here'
  }
})

[DougAnderson444]

Ok, great thanks Mauve! A few more late-night questions (I am very inquisitive):

the new one is for discovery-swarm-web servers.

Does/Will this discovery-swarm-web punch through the NAT well? Better/Worse/Same as going the hasbase route? Is there a public discovery-swarm-web server yet (similar to the gateway)?

There's two things you can do in this scenario: Either temporarily open it on a public gateway or add it to hashbase or a dat-store.

Got it. So dat-store is CLI, so to do this programmatically from the browser, looks like the best option is:

Use require('dat-registry') then:

• use (default) datbase.org, or;
• set opts.server =hashbase.io, or;
• setup one's own registry via homebase / hashbase ?

Do you think that would do the trick? I will give it a shot to see if I can get it working.

[DougAnderson444]

Or, setup dat-store on one's own server and call one's own API from the browser?

[RangerMauve]

discovery-swarm-web probably wont, but I'm working on a new thing called hyperswarm-proxy which is like the next generation of discovery-swarm-web which uses the new hyperswarm networking code which should be able to get through NAT.

The dat registry stuff is actually out of date. I use dat-pinning-service-client. You'll also need to make sure that your instance of dat-store has the proper CORS headers for the browser to be able to talk to it. (open an issue in dat-store if you have trouble with that)

[DougAnderson444]

Good to know! Thanks for the tip :)