search

storj-archived / core

Deprecated. Implementation of the Storj v2 protocol for Node.js.
https://storj.io
Other
395 stars 88 forks source link

Bridge minion are duplicating PUBLISH messages #258

Closed littleskunk closed 8 years ago

littleskunk commented 8 years ago

Package Versions

Replace the values below using the output from npm list storj.

"title":"Storj Bridge",
"version":"0.9.3",
"description":"Access the Storj network using a simple REST API.",
"x-protocol-version":"0.7.1",
"x-core-version":"1.4.0"

Replace the values below using the output from node --version.

I don't know.

Expected Behavior

Please describe the program's expected behavior. Include an example of your usage code in the back ticks below if applicable.

Bridge Server012 is uploading 453 files in one hour. Each file splitted in ~6 shards.

skunk@Server010:~$ grep 'PUSH' onehourbridgelog.log | wc -l
453
skunk@Server010:~$ grep '0f01020202' onehourbridgelog.log | wc -l
2369
skunk@Server010:~$ grep 'getting storage offer' onehourbridgelog.log | wc -l
2372

The plan was to send the contract PUBLISH to the nodeID close to the shard hash. 3 PUBLISH messages for each shard. I would expect 7116 PUBLISH messages.

Actual Behavior

Please describe the program's actual behavior. Please include any stack traces or log output in the back ticks below.

every of the 10 minions is sending the PUBLISH message. That would explain 71160 contract PUBLISH messages. 

skunk@Server010:~$ grep 'sending PUBLISH' onehourbridgelog.log | wc -l
76462

Even if the PUBLISH messages would be send to the nodeID close to the shard hash there is no reason to send it 10 times to the same nodeID.

Steps to Reproduce

Please include the steps the reproduce the issue, numbered below. Include as much detail as possible.

  1. Start bridge server and keep it running for a few hours
  2. Check the logfile. Ignore the first hour. We don't want to track the startup messages.
littleskunk commented 8 years ago

A little update based on this calculation: https://github.com/Storj/core/issues/250#issuecomment-234928305

It looks like the bridge is sending 30000-42000 tunnel PUBLISH messages. The best case would be ~42000 tunnel PUBLISH and ~5844 contract PUBLISH messages for 1838 shards. Looks perfect to me. Bridge is not duplicating the contract PUBLISH messages.

I will open a new issue for the 42000 tunnel PUBLISH messages. https://github.com/Storj/core/issues/283