NebulousLabs / skynet-js

A Javascript module made to simplify communication with Sia Skynet portals from the browser.
https://nebulouslabs.github.io/skynet-js/
MIT License
32 stars 12 forks source link

Modify SkyDB to optimize initial read/write times #304

Closed dghelm closed 3 years ago

dghelm commented 3 years ago

Current db.setJSON does the following in series:

  1. Tries to read if registry entry exists (5 second timeout)
  2. Upload Skyfile (~2 seonds)
  3. POSTs new registry entry
dghelm commented 3 years ago

Draft for option 1 at #305

mrcnski commented 3 years ago

Great job on option 1, that is an easy performance saving.

Can you provide an example for option 2 to help me understand it?

dghelm commented 3 years ago

I think Option 2 may be better as its own function, but here's the case for it.

Lots of new devs create apps where when a user logs in for the first time, a getJSON call is made looking for their application data that doesn't exist. Then if it doesn't, there's usually something written -- possibly another 5 seconds! The user is just left watching a spinner for 5-10 seconds, but this has the potential to cut to ~3 seconds.

Right now: Reading registry entries that don't exist is very slow, but writing to rev0 is very fast. Reading registry entries that do exist is very fast, but trying to write rev0 and waiting for it to fail is very slow.

What I want to do is pass an "initializeSkylink" to a method, it will then start three parallel actions:

  1. Will just try to write to rev0 using the supplied skylink
  2. Will do the normal "readRegistry" action to get a revision number
  3. Will start uploading the skyfile (unless its null)

Now: if 1 succeeds before 2 and a skyfile is provided, a registry write at rev1 can start. (2s upload + 150 ms registry write total) if 1 succeeds before 2 and no skyfile is provided, we're done and ready for fast registry reads on future setJSONs (100ms registry write total) if 2 succeeds before 1 and a skyfile is provided, a registry write at incremented rev can start (2s upload + 150ms registry write total) if 2 succeeds before 1 and a skyfile is not provided -- data already exists so we were already prepared for future setJSONs (150ms registry read total)

We may need a different method for this, but I'd like to hear thoughts.

mrcnski commented 3 years ago

Closing in flavor of https://github.com/SkynetHQ/skynet-js/issues/5