[RAD] I'm completely lost...

[rozek]

The title says everything: I'm completely lost...

I added RAD to my browser as mentioned in the docs and continued evaluating GUN - until I saw that there were only two keys in my "IndexedDB": namely "%1C" and "!"

So I started evaluating RAD using a trivialRADStore:

  const trivialRADStore_Store = Object.create(null)

  const trivialRADStore = {
    get: function (Key, Callback) {
      console.log('  trivialRADStore: reading', Key)
      return Callback(null,trivialRADStore_Store['' + Key])
    },

    put: function (Key, Data, Callback) {
      console.log('  trivialRADStore: writing', Key, '(' + typeof Data + ')')
      trivialRADStore_Store['' + Key] = Data
      return Callback(null,1)
    }
  }

which I could directly write to:

  const Rad = window.Radisk({ store:trivialRADStore })

  async function RAD_read (Key, Options) {
    return new Promise((resolve,reject) => {
      Rad(Key, (Error, Data, Info) => {
        console.log('  RAD Info:', Info)
        return (
          Error == null ? resolve(Data) : reject(Error)
        )
      }, Options)
    })
  }

  async function RAD_write (Key, Data) {
    return new Promise((resolve,reject) => {
      Rad(Key, Data, (Error, Data) => {
        return (
          Error == null ? resolve(Data) : reject(Error)
        )
      })
    })
  }

Surprisingly, my RAD adapter was also called with two keys only (the same as above). (However, I also logged the "info" record returned by the "read" variant of the RAD function - and that sometimes reported it would have parsed "1048576" bytes/chars although I had not written much at that point....)

Then I had the idea that I might have written too few keys and too little data - hence I wrote a simple function to fill my store

  async function writeNested (BaseKey,Depth) {
    for (let i = 0; i < 10; i++) {
      const currentKey = (BaseKey === '' ? '' : BaseKey + '/') + i
      await RAD_write(currentKey,currentKey)

      if (Depth > 1) { await writeNested(currentKey,Depth-1) }
    }
  }

and that soon started to take looooong to write a new entry - and had to be stopped after very few write calls (it was invoked with n = 3 - for 1110 entries - but did not come far...).

So: what am I doing wrong? Where is the "radix tree" promised by the docs?

[rozek]

a small note: if I change RAD_write and set the trivialRADStore_Store directly (i.e., avoid to call Rad), the whole loop completes almost immediately - thus, its RAD that cause the problems...

[rozek]

Ok, my current observations so far:

• entry "!" seems to collect written entries until they get stored away at their own keys
• I have no idea what entry "%1C" is good for - indeed I had to suppress reading and writing that entry! (see below)
• keys get encoded in a strange way: "1/1" becomes "1%2F1"
• there is no "radix tree" anywhere!
• entries are stored as JSON strings - but in a really strange format: writing "1/1" to key "1/1" (yes, in my experiments I used the key as value as well) creates an entry at "1%2F1" with the contents {"1/1":{"":"1/1"}}

In order to speed my in-memory(!) store up, I had to change my RAD adpter's code to

  const trivialRADStore_Store = Object.create(null)

  const trivialRADStore = {
    get: function (Key, Callback) {
      console.log('  trivialRADStore: reading', Key)
if (Key === '%1C') { return Callback(null,undefined) }
      return Callback(null,trivialRADStore_Store['' + Key])
    },

    put: function (Key, Data, Callback) {
      console.log('  trivialRADStore: writing', Key, typeof Data)
if (Key === '%1C') { return Callback(null,0) }
      trivialRADStore_Store['' + Key] = Data
      return Callback(null,1)
    }
  }

This is nothing but a hack - however, without that hack, my browser starts slowing down heavily after a few writes...

[rozek]

RAD also strangely reads the previously written key again - and writes it out again, unchanged! How useless is that?

[rozek]

I meanwhile set the chunk option to 1 for speedup

Here is a transcript of the first few operations on my in-memory store:

entry #1: 0
   trivialRADStore: reading %1C 0 undefined
   trivialRADStore: reading ! 0 undefined
   trivialRADStore: writing %1C 12 {"!":{"":1}}
   trivialRADStore: writing ! 20 {"0":{"":"value 0"}}
 entry #2: 0/0
   trivialRADStore: reading ! 20 {"0":{"":"value 0"}}
   trivialRADStore: writing %1C 25 {"!":{"":1},"0/0":{"":1}}
   trivialRADStore: writing 0%2F0 24 {"0/0":{"":"value 0/0"}}
   trivialRADStore: writing ! 20 {"0":{"":"value 0"}}
 entry #3: 0/1
   trivialRADStore: reading 0%2F0 24 {"0/0":{"":"value 0/0"}}
   trivialRADStore: writing %1C 41 {"!":{"":1},"0/":{"0":{"":1},"1":{"":1}}}
   trivialRADStore: writing 0%2F1 24 {"0/1":{"":"value 0/1"}}
   trivialRADStore: writing 0%2F0 24 {"0/0":{"":"value 0/0"}}
 entry #4: 0/2
   trivialRADStore: reading 0%2F1 24 {"0/1":{"":"value 0/1"}}
   trivialRADStore: writing %1C 52 {"!":{"":1},"0/":{"0":{"":1},"1":{"":1},"2":{"":1}}}
   trivialRADStore: writing 0%2F2 24 {"0/2":{"":"value 0/2"}}
   trivialRADStore: writing 0%2F1 24 {"0/1":{"":"value 0/1"}}

[rozek]

well, entry "%1C" seems to contain, what is called the "radix tree": the JSON string of an object with the hierarchy of keys.

I have no idea what that is good for as the get and put calls to my adapter always contain the full key of the affected entry...

[amark]

RAD being replaced with Book, just heads up. So if you're willing to help hack on that instead, please let me know! We do screen calls. @rogowski has been helping.

RAD chunks the tree into pages, so what you're seeing is the pages name and the root file that keeps track of the folder of files. More explained here:

https://youtu.be/5fCPRY-9hkc?t=1275

but note: Book will be replacing this, so don't spend too much time/effort on it.

[rozek]

That's good to know - I already lost a day on this because I searched the problem on my side...will give up on RAD then!

[jadbox]

@amark Where can I find information on Book?

[amark]

@rozek I'm back to working on Book, was distracted with @SecureRender for a while. Want to help me & Rogowski?

@jadbox same??? Sorry not much docs yet, mostly just ask me on http://chat.gun.eco and maybe look thru tests in tests/rad/book (or... somewhere like that) or jump on a call with me (DM on twitter).