ssbc / go-ssb

Go implementation of ssb (work in progress!)
https://scuttlebutt.nz
165 stars 23 forks source link

Where are the indexes actually being set? #306

Open KyleMaas opened 1 year ago

KyleMaas commented 1 year ago

From what I can tell from reading through the Margaret code, there should be some kind of a processing/setter function which is called by Margaret for each message on each index which either sets a Margaret index value or doesn't, depending on whether the message matches what's needed for that index. Where are these filter/set functions?

KyleMaas commented 1 year ago

The only places I'm seeing indexes actually having a Set() called on them is within plugins, and none of them look like they set any values that would be queried by, for example, the ByType index or the Users index.

KyleMaas commented 1 year ago

I think most (but still not all) of this is being handled here:

https://github.com/ssbc/go-ssb/blob/master/multilogs/combined.go

boreq commented 1 year ago

I think most (but still not all) of this is being handled here:

https://github.com/ssbc/go-ssb/blob/master/multilogs/combined.go

That is just the combined index which isn't really used that much. The actual updates to other indexes happen for example here:

https://github.com/ssbc/go-ssb/blob/55f48949535cc3e42ec8cbf20b75e1cb003a7ab9/graph/builder_indexing.go#L103

KyleMaas commented 1 year ago

@boreq Shoot. That system has a bunch of possible race condition paths and is not well-synchronized with sbot, as I discovered when fixing TestPeople. I'll have to do some more work on that. Thanks!

KyleMaas commented 1 year ago

@boreq Thanks for the pointer - see #323, where I tied those indexes into sbot.