gatherer_p2p 1.1.0

[theoden-dd]

• 1.1.0

  • Duplicate exclusion after random selection. Now every broadcasting, gathering and receiving marks a node as sent. Sent nodes won't be resent until the next session (addon reloading). The purpose is to send out the database as fast as possible. https://github.com/theoden-dd/gatherer_p2p/issues/7
  • Initial delay was set to 4 seconds. Due to sent miss throttling, its limit tends to 15 seconds That's even more, than previous 10 seconds.
  • Raid/party broadcasting. https://github.com/theoden-dd/gatherer_p2p/issues/8
  • Option "p2p" became "guild" and toggles broadcasting to guild.
  • Option "raid" toggles broadcasting to raid/party and defaults to off. / Incendicite Ore will now be ignored and will not produce an error.

• 1.0.2 - hotfix / bug with random selection from empty database https://github.com/theoden-dd/gatherer_p2p/issues/14

• 1.0.1 - hotfix / RuWow addon incompatibility. https://github.com/theoden-dd/gatherer_p2p/issues/5

  • locale option to override locale. This is a hack to overcome RuWow's messing with the locale.

[jsb]

Here's my understanding of the "sent skipping" mechanism:

• At every cycle, select a random node.
  • If it has already been sent, do nothing and increase the cycle frequency.
  • If it has not been sent yet, broadcast the data and reset the cycle frequency to default.

Is that correct?

I see the following issue with this: What happens after all nodes known to you have been marked as "sent" (might happen for new users, or after playing for a long time)? This would cause the cycle frequency to increase forever, (possibly deadlocking the client?) and no nodes be broadcast anymore.

Ideally, the selectRandomGather function would be clever enough to always return a node that hasn't been marked as "sent" yet (or, if that isn't possible, just clear the "sent" list and start over). That would eliminate the need for the cycle frequency adjustment and skipped_cycles_count bookkeeping. However, the only obvious way I see to do this (i.e., scanning the entire database for "unsent" items every time) sounds potentially slow.

What do you think?

[theoden-dd]

You're right at every point.

In practice, I was unable to play long enough to have perfomance issues. After 4-5 hours of gameplay I had got 50-100 skips per sending and 75% of db marked as sent. All numbers are rough approximates.

Exhausting database fully and falling into the endless skipping could be a case only for a really small databases.

Actually I relied on average scenario: medium to large db. Thank you for paying attention to the small sized db edge case. Anyway, there is workaround with turning off node exchanging while the db is small.

I'll made a test with a single node database and answer on results.

I intend to address both flaws you named in the next 1.2.0 version. See theoden-dd#13 for the random selection always returning a node. The "all sent" solution is way simpler: I'll empty the sent table after it is full. We've sent our db fully - so we could restart from scratch!

Don't think I've decided to release a piece of crap. I still consider current version good enough for usage. Well, I'm actually using it by myself along with several friends. Also I consider it a way better than a previous, treating the fact it'll not resend a node twice during a session as a local killer-feature.

[theoden-dd]

Thank you, the experiment was very interesting experience.

First, there was a bug that led to a constant delay of 4 seconds. Second, after I've fixed it, there were no client freezes even after 300K skips. After that evidence I've remembered that function onUpdate is capped by fps, anyway. So frequency was de facto no more, than 50-200 Hz.

However, reselecting items 50-200 times per second in a single node database is rather dumb cpu load, you're right.

So I've made a hotfix with resetting sent marks.

As a result on a single node database it exhausts db every 4 seconds and congratulates user he sent out his full db.

Closing this now. I'll open a new pull-request for the hotfix.