tournesol-app / tournesol

Free and open source code of the https://tournesol.app platform. Meet the community on Discord https://discord.gg/WvcSG55Bf3
https://tournesol.app
Other
325 stars 47 forks source link

Improve the performance of the vouching algorithm #1345

Open glerzing opened 1 year ago

glerzing commented 1 year ago

Why do we consider all the users in the vouching matrix ?

If we had 100,000 contributors, we could not implement such a large matrix (if each element is coded on a 4-bytes float, that would use (100,000)² * 4 = 40 Gb of data).

A practical workaround would be to consider only the users that have pretrust or have been vouched, which would reduce the size of the matrix and the time complexity of the calculations by a lot.

lenhoanglnh commented 1 year ago

I think that it's possible to only track vouches, yielding a space and time complexity of O(#contributors + #vouches). Essentially, this is because the iteration algorithm for trusts merely consists of propagating trusts along the vouches of the network.