search

storj-archived / core

Deprecated. Implementation of the Storj v2 protocol for Node.js.
https://storj.io
Other
395 stars 88 forks source link

Farmer crashes after daemon 4.0.0/4.0.1 upgrade (which includes v7.0.0 of core) #720

Closed felixbrucker closed 7 years ago

felixbrucker commented 7 years ago

Package Versions

Replace the values below using the output from npm list storj. Use npm list -g storj if installed globally.

storjshare-daemon: 4.0.1
core: 7.0.0

Replace the values below using the output from node --version.

v6.11.2

Expected Behavior

Please describe the program's expected behavior. Include an example of your usage code in the back ticks below if applicable.

The farmer doesn't crash.

Actual Behavior

Please describe the program's actual behavior. Please include any stack traces or log output in the back ticks below.

The farmer crashes (randomly), but only one node seems to be affected. Other nodes are in different geolocations.

/home/felix/.nvm/versions/node/v6.11.2/lib/node_modules/storjshare-daemon/node_modules/storj-lib/lib/network/farmer.js:447
  const sigObj = secp256k1.sign(sighash, privkey);
                           ^

RangeError: private key length is invalid
    at RangeError (native)
    at FarmerInterface.bridgeRequest (/home/felix/.nvm/versions/node/v6.11.2/lib/node_modules/storjshare-daemon/node_modules/storj-lib/lib/network/farmer.js:447:28)
    at ShardServer._sendExchangeReport (/home/felix/.nvm/versions/node/v6.11.2/lib/node_modules/storjshare-daemon/node_modules/storj-lib/lib/network/shard-server.js:150:24)
    at ReadableFileStream._handleTransferFinish (/home/felix/.nvm/versions/node/v6.11.2/lib/node_modules/storjshare-daemon/node_modules/storj-lib/lib/network/shard-server.js:312:12)
    at emitNone (events.js:91:20)
    at ReadableFileStream.emit (events.js:185:7)
    at endReadableNT (/home/felix/.nvm/versions/node/v6.11.2/lib/node_modules/storjshare-daemon/node_modules/readable-stream/lib/_stream_readable.js:992:12)
    at _combinedTickCallback (internal/process/next_tick.js:80:11)
    at process._tickDomainCallback (internal/process/next_tick.js:128:9)

Steps to Reproduce

Please include the steps the reproduce the issue, numbered below. Include as much detail as possible.

I have not yet found a way to reproduce the problem.

him2him2 commented 7 years ago

I was also having this problem after upgrading. I have 6 nodes on that box but only one was having this problem. It was so bad that the node stopped after 30 restart re-trys.

I thought I was able to resolve this problem by stopping all nodes:

storjshare-killall

Then re-trying the upgrade:

npm install --global storjshare-daemon

Then restarting all the nodes.

All nodes have been running with no issues for 10 mins. Will keep an eye on it.

Update: After 1hr 41mins the node restarted. It has restarted a total of 4 times in the last 10 minutes. Storage for the node had grown by 130 MB and everything else looked ok. Same error when it restarted. 

/usr/lib/node_modules/storjshare-daemon/node_modules/storj-lib/lib/network/farmer.js:447
  const sigObj = secp256k1.sign(sighash, privkey);
                           ^

RangeError: private key length is invalid
    at FarmerInterface.bridgeRequest (/usr/lib/node_modules/storjshare-daemon/node_modules/storj-lib/lib/network/farmer.js:447:28)
    at ShardServer._sendExchangeReport (/usr/lib/node_modules/storjshare-daemon/node_modules/storj-lib/lib/network/shard-server.js:150:24)
    at ReadableFileStream._handleTransferFinish (/usr/lib/node_modules/storjshare-daemon/node_modules/storj-lib/lib/network/shard-server.js:312:12)
    at emitNone (events.js:91:20)
    at ReadableFileStream.emit (events.js:188:7)
    at endReadableNT (/usr/lib/node_modules/storjshare-daemon/node_modules/readable-stream/lib/_stream_readable.js:992:12)
    at _combinedTickCallback (internal/process/next_tick.js:80:11)
    at process._tickDomainCallback (internal/process/next_tick.js:128:9)
{"level":"error","message":"failed to relay publication to random neighbor {\"userAgent\":\"6.8.0\",\"protocol\":\"1.1.0\",\"address\":\"75.158.76.73\",\"port\":63201,\"nodeID\":\"f491e38ce1e2a6b6db78686f5700bfbdb0cc30a6\",\"lastSeen\$
{"level":"error","message":"failed to relay publication to random neighbor {\"userAgent\":\"6.8.0\",\"protocol\":\"1.1.0\",\"address\":\"75.158.76.73\",\"port\":63201,\"nodeID\":\"f491e38ce1e2a6b6db78686f5700bfbdb0cc30a6\",\"lastSeen\$
{"level":"error","message":"failed to relay publication to random neighbor {\"userAgent\":\"6.8.0\",\"protocol\":\"1.1.0\",\"address\":\"75.158.76.73\",\"port\":63201,\"nodeID\":\"f491e38ce1e2a6b6db78686f5700bfbdb0cc30a6\",\"lastSeen\$
{"level":"error","message":"failed to relay publication to random neighbor {\"userAgent\":\"6.8.0\",\"protocol\":\"1.1.0\",\"address\":\"218.206.79.93\",\"port\":40002,\"nodeID\":\"f4f60a7f58126c437878da71f52ce9af4ece592b\",\"lastSeen$
{"level":"error","message":"failed to relay publication to random neighbor {\"userAgent\":\"6.8.0\",\"protocol\":\"1.1.0\",\"address\":\"realshels.zapto.org\",\"port\":4120,\"nodeID\":\"f4df03233b3d7e50add087e17607cbc8d96535b6\",\"las$
{"level":"error","message":"failed to relay publication to random neighbor {\"userAgent\":\"6.8.0\",\"protocol\":\"1.1.0\",\"address\":\"75.158.76.73\",\"port\":63201,\"nodeID\":\"f491e38ce1e2a6b6db78686f5700bfbdb0cc30a6\",\"lastSeen\$
{"level":"error","message":"failed to relay publication to random neighbor {\"userAgent\":\"6.8.0\",\"protocol\":\"1.1.0\",\"address\":\"realshels.zapto.org\",\"port\":4120,\"nodeID\":\"f4df03233b3d7e50add087e17607cbc8d96535b6\",\"las$
{"level":"error","message":"failed to relay publication to random neighbor {\"userAgent\":\"6.8.0\",\"protocol\":\"1.1.0\",\"address\":\"realshels.zapto.org\",\"port\":4120,\"nodeID\":\"f4df03233b3d7e50add087e17607cbc8d96535b6\",\"las$
{"level":"error","message":"failed to relay publication to random neighbor {\"userAgent\":\"6.5.0\",\"protocol\":\"1.1.0\",\"hdKey\":\"xpub6AHweYHAxk1EhJSBctQD1nLWPog6Sy2eTpKQLExR1hfzTyyZQWvU4EYNXv1NJN7GpLYXnDLt4PzN874g6zSjAQdFCHZN7U7$
{"level":"error","message":"failed to relay publication to random neighbor {\"userAgent\":\"6.7.0\",\"protocol\":\"1.1.0\",\"address\":\"5.9.150.177\",\"port\":4200,\"nodeID\":\"f4cd9d26afad18b790db7e49b03ebf5e427537c9\",\"lastSeen\":$
{"level":"error","message":"failed to relay publication to random neighbor {\"userAgent\":\"6.8.0\",\"protocol\":\"1.1.0\",\"address\":\"91.218.249.43\",\"port\":4000,\"nodeID\":\"f4e78f2f07532350958b27f3d94c30d6bf1d9ba8\",\"lastSeen\$
{"level":"error","message":"failed to relay publication to random neighbor {\"userAgent\":\"6.8.0\",\"protocol\":\"1.1.0\",\"address\":\"104.168.62.178\",\"port\":4000,\"nodeID\":\"f4fd47716ecfdb1eaedf54a4f41fb01f5baf3086\",\"lastSeen$
{"level":"error","message":"failed to relay publication to random neighbor {\"userAgent\":\"6.8.0\",\"protocol\":\"1.1.0\",\"address\":\"35.195.166.48\",\"port\":48407,\"nodeID\":\"f6949f5fbf8758807416c1a2e2e93ea316641ce5\",\"lastSeen$
{"level":"error","message":"failed to relay publication to random neighbor {\"userAgent\":\"6.8.0\",\"protocol\":\"1.1.0\",\"address\":\"35.195.166.48\",\"port\":48407,\"nodeID\":\"f6949f5fbf8758807416c1a2e2e93ea316641ce5\",\"lastSeen$
{"level":"error","message":"failed to relay publication to random neighbor {\"userAgent\":\"6.8.0\",\"protocol\":\"1.1.0\",\"address\":\"45.32.72.183\",\"port\":4000,\"nodeID\":\"f5957d2ed5cae32efed105bd371320c3f047e242\",\"lastSeen\"$
aleitner commented 7 years ago

If you can get it, what's the length of your private keys? Have a feeling if it starts with two 0s then it chops off the first one

braydonf commented 7 years ago

Found the issue, will have PR for it shortly.

felixbrucker commented 7 years ago

i have taken a look at the private key in question, it does not start with any zeros:

f263cd7a83f455ab5fb54fe74e81f1c50f7f83fc45dedd23f00b860a37f5f3

i have changed one character in the middle, but i really do not care if somebody where to find the real key, i do not use it anymore.

i have not yet tried running that share yet again

Edit: I have taken a closer look and the key is in fact two chars too short. Possibly the share creation already skipped the zeros so there weren't any in the config file to begin with, then of course this fix will apply. Thanks

braydonf commented 7 years ago

Yep, there are two leading zeros for that, the length is 62, when it should be 64. It's how the big number for the key is made into a string. The logic for why it's not padded is because we don't usually write 000010 and 000011, however with private keys the length is supposed to always be the same at 32 bytes.