ssbc / ssb-fixtures

Generate a simulated .ssb folder
10 stars 1 forks source link

Generate contact messages according to a passed-in follow graph #5

Closed cblgh closed 3 years ago

cblgh commented 3 years ago

hi hi!

as you are aware, mr @staltz, we are coworkers atm—and i am currently starting up work on a network simulator as part of the ngi work.

one of the considerations i am dealing with right now is where/how/if to use ssb-fixtures data wrt the sim. one concern on my mind: i think i want to allow network simulator users to specify their own network topology, i.e. the hops structure tying together different ssb users. currently, if i was to use the fixtures data, that would be hard; the fixtures would define the network topology instead, and i'm not sure yet if that is what i want.

thus, the ask: would it be difficult to generate contact (follow) messages in ssb-fixtures according to a passed-in follow graph? you can choose the format you think is best, e.g.

{
  "<feedidx>": ["feedid1", "feedid2"],
  "<feedidy>": ["feedid1", "feedid3"]
}

the prime difficulty i see is associating the identifiers in the passed-in graph with the feed identifiers generated by ssb-fixtures; but the specific identifiers in the graph don't matter as much as the relationship between the feeds (and of course knowing how the graph identifiers map to the generated identifiers)

staltz commented 3 years ago

I think it would be a lot easier to do the opposite: use ssb-fixtures to generate a bunch of data, extract the follow graph from that, and then use the follow graph to boot up netsim. the individual peers in netsim could "bootstrap" their ~/.ssb folder by partially replicating the origin ssb-fixture dataset (note that this "partial replication" has nothing to do with "secure partial replication", i'm talking about just a way of picking a subset of the dataset for each node in the netsim, so they have a starting point and that starting point isn't the same for all).

thus, the ask: would it be difficult to generate contact (follow) messages in ssb-fixtures according to a passed-in follow graph?

rather difficult. I mean, it's possible, but requires working on ssb-fixtures quite a bit.

the prime difficulty i see is associating the identifiers in the passed-in graph with the feed identifiers generated by ssb-fixtures

this is another sign why it's best to use the feed IDs from ssb-fixtures, because the responsibility of ssb-fixtures is precisely with generating that data to begin with. The important thing about ssb-fixtures and how it does generation is that the follow graph is built with statistical sampling that imitates real world patterns of connection. see https://github.com/ssb-ngi-pointer/ssb-fixtures/blob/main/src/frequencies.ts

if you pass in a follow-graph, it won't have these real world statistical properties. if it would, then it would be doing the same responsibility as ssb-fixtures

cblgh commented 3 years ago

(for a follow-up on this question & initial proposal, see the network simulator draft :3)