ssbc / ssb-social-index

scuttlebutt plugin for getting reduced state based on the author's social graph
5 stars 2 forks source link

Support db2 #4

Closed arj03 closed 3 years ago

arj03 commented 3 years ago

This PR should allow this module to use db2 queries instead of backlinks queries if db2 has been loaded as a secret stack plugin.

Totally untested so this might not work at all ;-)

@KyleMaas you should be able to use this module to get names, images etc. instead of using getProfile. Note the API is a bit different because multiple people can assign about messages for something where ssb-browser before would just take the authors own values. Also this module is used by ssb-about.

This might need some caching to be really performant, but first step is just getting it working :)

Also this needs a README update, but first things first.

KyleMaas commented 3 years ago

This requires ssb-backlinks, which doesn't support db2 yet:

https://github.com/ssbc/ssb-backlinks/issues/22

KyleMaas commented 3 years ago

Okay, cool. Was just reading through the documentation to try to figure out how to use these properly and noticed that as a dependency.

KyleMaas commented 3 years ago

I guess that would be where the README update would come in. :)

KyleMaas commented 3 years ago

First issue: socialValue() crashes here:

https://github.com/ssbc/ssb-social-index/blob/6a5883aba6d2d0b155d7bf033d10e62fd9375105/index.js#L250

Called from here:

https://github.com/ssbc/ssb-social-index/blob/6a5883aba6d2d0b155d7bf033d10e62fd9375105/index.js#L172

TypeError: live is not a function
    at read (bundle-ui.js:formatted:65053)
    at socialValues (bundle-ui.js:formatted:64967)
    at bundle-ui.js:formatted:64851
    at getAuthor (bundle-ui.js:formatted:65078)
    at Object.socialValue (bundle-ui.js:formatted:64848)
    at Object.hooked [as socialValue] (bundle-ui.js:formatted:6859)
    at Object.SSB.getProfileNameAsync (bundle-ui.js:formatted:73264)
    at a.created (bundle-ui.js:formatted:166298)
    at He (vue.min.js:6)
    at Yt (vue.min.js:6)
arj03 commented 3 years ago

Right, needs to be liveOp ? liveOp(liveOpts) : null, ;-)

KyleMaas commented 3 years ago

I figured. Did you want to change that or do you want to me to just get it all working and file a pull request against this branch?

arj03 commented 3 years ago

I think its easier if you just create a new PR based on this once things are working.

arj03 commented 3 years ago

And its great that you are asking questions instead of getting stuck :)

KyleMaas commented 3 years ago

Okay. That may be for the best. Running into some other issues as well.

KyleMaas commented 3 years ago

Couple more issues:

  1. These guys here: https://github.com/ssbc/ssb-social-index/blob/6a5883aba6d2d0b155d7bf033d10e62fd9375105/index.js#L226 ...give me errors about bValue, bContent, and bAbout being uninitalized. Replacing them with Buffer.from('value'), etc. fixes those. So there's some kind of a scope error there.

  2. If it doesn't find any records, the callback passed to socialValue() never gets called, so you have no idea that there were no values to get.

  3. I can't seem to get it to actually return any values at all, even if I make the query nothing more than and(type(options.type)) with options.type set to "about"

arj03 commented 3 years ago
1. These guys here:
   https://github.com/ssbc/ssb-social-index/blob/6a5883aba6d2d0b155d7bf033d10e62fd9375105/index.js#L226

   ...give me errors about bValue, bContent, and bAbout being uninitalized.  Replacing them with Buffer.from('value'), etc. fixes those.  So there's some kind of a scope error there.

Strange. It should work. The performance difference is probably not that bad including them directly in the function instead.

2. If it doesn't find any records, the callback passed to socialValue() never gets called, so you have no idea that there were no values to get.

Strange. I wonder if that is a general issue?

3. I can't seem to get it to actually return any values at all, even if I make the query nothing more than `and(type(options.type))` with options.type set to "about"

Strange. I did actually run that query outside of this module to see if it was correct. Is it because live is set and old false?

KyleMaas commented 3 years ago
  1. It's an easy enough fix if that works.
  2. Since I haven't gotten anything to work yet, I can't speak to this.
  3. Even if I add liveOp({ old: true}) to the query, it still doesn't work.
KyleMaas commented 3 years ago

How well-tested is this module without this pull request? Because, inside of socialValues(), I can get read() to return values which are fed to pull.drain(), which tries to add them to values[], but it never reaches the (err) => {} section to call the callback. That didn't change from the old version to this version, so I'm wondering if this ever actually worked.

KyleMaas commented 3 years ago

Also, the only way I can get read() to actually get any records is if I leave off the about() part and limit only by type.

arj03 commented 3 years ago

I got it more or less working. There seems to be a problem with live. Checking that out now.

arj03 commented 3 years ago

There, liveOpts and live was switched ;)

arj03 commented 3 years ago

Just remembered that it might be good to use prefix: 32 in the equal operator so it can create the index only once.

KyleMaas commented 3 years ago

I still can't get that to actually return anything. Here's what I've got going right now:

https://github.com/KyleMaas/ssb-browser-core/tree/new-ssb-plugins

https://github.com/KyleMaas/ssb-browser-demo/tree/new-ssb-plugins

(Please note that ssb-browser-core is missing the dependency on ssb-social-index in package.json because it runs off of a local checkout.)

Every time I try to get a name from it, it outputs "Falling back to getProfileAsync", which indicates that it's not getting a result back. What am I doing wrong?

arj03 commented 3 years ago

I see the problem now. The problem is that at first there is no index for this new about and you get a ton of requests and they all start rebuilding the index concurrently. Really sorry to let you down the deep end here Kyle. This problem needs to be fixed in jitdb, I'll let you know when its fixed and you can test more :)

KyleMaas commented 3 years ago

Well, I feel a little better about that, then. :)

arj03 commented 3 years ago

@KyleMaas we fixed the problems that was causing this to be slow. Try this against latest ssb-browser-core.

KyleMaas commented 3 years ago

Hey, that works! Nice!

KyleMaas commented 3 years ago

Pushed changes to both ssb-browser-core and ssb-browser-demo under new-ssb-plugins branches if you want to try them. Same thing as before - I used local git clones, so the dependencies are probably a little goofy. Are these changes in the mainline packages where I could just pull them in using normal npm now?

KyleMaas commented 3 years ago

Appears to be working with mainline npm install.

arj03 commented 3 years ago

I think this should be ready now. Seems to be pretty well in browser and in ssb-suggest. I was thinking of doing a 2.0 release when this is merged.

staltz commented 3 years ago

I think this should be ready now. Seems to be pretty well in browser and in ssb-suggest.

Yep, I looked at it 3 times and can't find problems.

I was thinking of doing a 2.0 release when this is merged.

If we're strict about semver, there's no breaking change here, so it should be 1.1.0.