Closed AlexSSD7 closed 4 years ago
Let's assume that the eth2 network has 5000 validating nodes out there, and the average internet speed worldwide is 11 mbps.
To be sure that the node would be DDoS'ed - we will dedicate 22 mbps (2x speed) to each one.
Note that in real world speed would be adjusted for every node by determining the ISP's average speed (is it a home ISP, hosting, business, etc).
DigitalOcean's average droplet speed is 300mbps, so it should be enough to just rent 5000*22/300 = 36 cheapest droplets ($5 a month) to perform a massive attack.
So the total price of a day doing the attack is 36*5/30.5 = $5.90.
Please note that this is just an example. A real world attack would look a bit different.
Hey! Thanks for reaching out. At the moment we are looking into some other attacknet issues, so responses may be slower than normal.
Some quick feedback: At 300mbps, for a month: 300 * 10e6 * 30 * 24 * 60 * 60 / 8 / 10e9 = 97200 GB
of data, per month. The digital ocean pricing calculator says 5 USD per month. But almost 1k USD in bandwidth costs (only accounting for outwards traffic here). Now times 36 to have more nodes do the same. It's not completely out of reach, but not as cheap as a mere 6 dollars. With some more hardening in prysm, to make it less effective, cost would increase as well.
Yeah, you're right. Obviously, no one would use digitalocean to perform DDoS attacks because they can suspend the account.
I used DigitalOcean to simulate the botnet that black hats would have used.
Update: the network was unfinalized for 38 epochs despite I have stopped the attack on ~20 unfinalized epochs. Some validators are down too.
To update we had block proposals timing out for an extended amount of time
Beacon Node:
time="2020-07-29 02:07:25" level=error msg="Failed to close stream" error="read tcp4 172.31.2.203:9000->69.178.221.103:1024: read: connection reset by peer" prefix=sync
time="2020-07-29 02:52:25" level=error msg="Failed to close stream" error="read tcp4 172.31.2.203:9000->82.64.27.147:13000: read: connection reset by peer" prefix=sync
time="2020-07-29 03:10:25" level=error msg="Failed to close stream" error="read tcp4 172.31.2.203:9000->54.87.247.218:17076: read: connection reset by peer" prefix=sync
time="2020-07-29 06:16:50" level=error msg="Failed to close stream" error="read tcp4 172.31.2.203:9000->82.64.27.147:13000: read: connection reset by peer" prefix=sync
time="2020-07-29 06:20:36" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 08:21:50" level=error msg="Failed to revalidate peer" error="i/o deadline exceeded" peer=16Uiu2HAmNNrpDyCHqtRGU9CnvHuwGaRBaTLN8HffA226MnsK4n2A prefix=sync
time="2020-07-29 08:25:02" level=error msg="Failed to revalidate peer" error="i/o deadline exceeded" peer=16Uiu2HAmNNrpDyCHqtRGU9CnvHuwGaRBaTLN8HffA226MnsK4n2A prefix=sync
time="2020-07-29 08:28:14" level=error msg="Failed to revalidate peer" error="i/o deadline exceeded" peer=16Uiu2HAmNNrpDyCHqtRGU9CnvHuwGaRBaTLN8HffA226MnsK4n2A prefix=sync
time="2020-07-29 08:31:26" level=error msg="Failed to revalidate peer" error="i/o deadline exceeded" peer=16Uiu2HAmNNrpDyCHqtRGU9CnvHuwGaRBaTLN8HffA226MnsK4n2A prefix=sync
time="2020-07-29 08:53:50" level=error msg="Failed to revalidate peer" error="i/o deadline reached" peer=16Uiu2HAkxvVfxCgL9oUJG3uULpq9rEwzxEcVBzgix6maxe7oXtyV prefix=sync
time="2020-07-29 09:06:38" level=error msg="Failed to revalidate peer" error="i/o deadline reached" peer=16Uiu2HAmVssyKiSKoSekvb35AFeFB8jEBsDVYx4U8r3NJVc2NZii prefix=sync
time="2020-07-29 09:07:12" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 09:07:24" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 09:07:48" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 09:08:00" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 09:09:00" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 09:15:00" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 09:18:48" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 09:21:12" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 09:22:38" level=error msg="Failed to revalidate peer" error="i/o deadline reached" peer=16Uiu2HAmBDM7zQgPhYLR4W97a4CeQR9wNLymXLQyekT7hz8RN1e5 prefix=sync
time="2020-07-29 09:23:24" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 09:24:00" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 09:24:48" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 09:25:00" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 09:29:48" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 09:32:12" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 09:32:24" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 09:35:00" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 09:35:36" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 09:38:24" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 09:38:36" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 09:44:12" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 09:44:36" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 09:44:57" level=error msg="Failed to revalidate peer" error=EOF peer=16Uiu2HAmRt42NX7EHMGVRPdLmcwtdSye9ezdTh2KUXeaKQWGT8Gn prefix=sync
time="2020-07-29 09:45:24" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 09:45:48" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 09:47:00" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 09:47:36" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 09:48:10" level=error msg="Failed to revalidate peer" error=EOF peer=16Uiu2HAmRt42NX7EHMGVRPdLmcwtdSye9ezdTh2KUXeaKQWGT8Gn prefix=sync
time="2020-07-29 09:48:10" level=error msg="Failed to revalidate peer" error=EOF peer=16Uiu2HAmDmNWARMgdSNAq29hcGU9pmFMxWtMNR65ch9QfWj12KKY prefix=sync
time="2020-07-29 09:48:13" level=error msg="Failed to revalidate peer" error=EOF peer=16Uiu2HAmJaGbUaHdxAHduoMpEmEemcYUx4CXAJzryd5A8sLv4j6Y prefix=sync
time="2020-07-29 09:48:14" level=error msg="Failed to revalidate peer" error="i/o deadline reached" peer=16Uiu2HAmVssyKiSKoSekvb35AFeFB8jEBsDVYx4U8r3NJVc2NZii prefix=sync
time="2020-07-29 09:48:24" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 09:49:48" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 09:50:12" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 09:51:22" level=error msg="Failed to revalidate peer" error=EOF peer=16Uiu2HAmDmNWARMgdSNAq29hcGU9pmFMxWtMNR65ch9QfWj12KKY prefix=sync
time="2020-07-29 09:51:22" level=error msg="Failed to revalidate peer" error=EOF peer=16Uiu2HAmRt42NX7EHMGVRPdLmcwtdSye9ezdTh2KUXeaKQWGT8Gn prefix=sync
time="2020-07-29 09:51:23" level=error msg="Failed to revalidate peer" error=EOF peer=16Uiu2HAmJaGbUaHdxAHduoMpEmEemcYUx4CXAJzryd5A8sLv4j6Y prefix=sync
time="2020-07-29 09:51:26" level=error msg="Failed to revalidate peer" error="i/o deadline reached" peer=16Uiu2HAmVssyKiSKoSekvb35AFeFB8jEBsDVYx4U8r3NJVc2NZii prefix=sync
time="2020-07-29 09:54:34" level=error msg="Failed to revalidate peer" error="wrong fork digest version" peer=16Uiu2HAmJaGbUaHdxAHduoMpEmEemcYUx4CXAJzryd5A8sLv4j6Y prefix=sync
time="2020-07-29 09:54:34" level=error msg="Failed to revalidate peer" error=EOF peer=16Uiu2HAmRt42NX7EHMGVRPdLmcwtdSye9ezdTh2KUXeaKQWGT8Gn prefix=sync
time="2020-07-29 09:54:35" level=error msg="Failed to revalidate peer" error=EOF peer=16Uiu2HAmDmNWARMgdSNAq29hcGU9pmFMxWtMNR65ch9QfWj12KKY prefix=sync
time="2020-07-29 09:55:36" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 09:57:46" level=error msg="Failed to revalidate peer" error="wrong fork digest version" peer=16Uiu2HAmDmNWARMgdSNAq29hcGU9pmFMxWtMNR65ch9QfWj12KKY prefix=sync
time="2020-07-29 09:57:48" level=error msg="Failed to revalidate peer" error=EOF peer=16Uiu2HAmRt42NX7EHMGVRPdLmcwtdSye9ezdTh2KUXeaKQWGT8Gn prefix=sync
time="2020-07-29 09:59:00" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 10:01:48" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 10:03:00" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 10:04:14" level=error msg="Failed to revalidate peer" error="i/o deadline reached" peer=16Uiu2HAmVssyKiSKoSekvb35AFeFB8jEBsDVYx4U8r3NJVc2NZii prefix=sync
time="2020-07-29 10:04:48" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 10:07:48" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 10:14:48" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 10:15:00" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 10:16:12" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 10:18:00" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 10:20:12" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 10:21:36" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 10:22:24" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 10:27:48" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 10:28:12" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 10:28:48" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 10:34:00" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 10:36:00" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 10:36:14" level=error msg="Failed to revalidate peer" error="i/o deadline reached" peer=16Uiu2HAmVssyKiSKoSekvb35AFeFB8jEBsDVYx4U8r3NJVc2NZii prefix=sync
time="2020-07-29 10:38:48" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 10:39:36" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 10:42:12" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 10:43:55" level=error msg="Could not get rough time result: lookup roughtime.chainpoint.org on 172.31.0.2:53: read udp 172.31.2.203:38221->172.31.0.2:53: i/o timeout" prefix=roughtime
time="2020-07-29 10:43:55" level=error msg="Could not get rough time result: no reply" prefix=roughtime
time="2020-07-29 10:47:24" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 10:49:12" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 10:51:36" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 10:52:10" level=error msg="Failed to revalidate peer" error=EOF peer=16Uiu2HAmRt42NX7EHMGVRPdLmcwtdSye9ezdTh2KUXeaKQWGT8Gn prefix=sync
time="2020-07-29 10:52:10" level=error msg="Failed to revalidate peer" error=EOF peer=16Uiu2HAm7ukQsQ3Vx9ButibFzQ6eCJBnDDpe6QEMtfQKpXetexYq prefix=sync
time="2020-07-29 10:52:13" level=error msg="Failed to revalidate peer" error="wrong fork digest version" peer=16Uiu2HAmKb72XTXHBYYsBYWYsWuE5AeBeFw2awt518cWrwh4Anvf prefix=sync
time="2020-07-29 10:52:36" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 10:53:36" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 10:54:24" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 10:54:36" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 10:55:23" level=error msg="Failed to revalidate peer" error=EOF peer=16Uiu2HAm7ukQsQ3Vx9ButibFzQ6eCJBnDDpe6QEMtfQKpXetexYq prefix=sync
time="2020-07-29 10:56:00" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 10:58:12" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 10:58:34" level=error msg="Failed to revalidate peer" error=EOF peer=16Uiu2HAm7ukQsQ3Vx9ButibFzQ6eCJBnDDpe6QEMtfQKpXetexYq prefix=sync
time="2020-07-29 11:00:12" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 11:01:46" level=error msg="Failed to revalidate peer" error=EOF peer=16Uiu2HAm7ukQsQ3Vx9ButibFzQ6eCJBnDDpe6QEMtfQKpXetexYq prefix=sync
time="2020-07-29 11:02:36" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 11:03:12" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 11:03:48" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 11:04:58" level=error msg="Failed to revalidate peer" error=EOF peer=16Uiu2HAm7ukQsQ3Vx9ButibFzQ6eCJBnDDpe6QEMtfQKpXetexYq prefix=sync
time="2020-07-29 11:06:24" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 11:11:00" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 11:16:12" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 11:20:00" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 11:20:24" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 11:27:36" level=error msg="Failed to get block number from timestamp" error="context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 11:31:36" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 11:38:00" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 11:38:48" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 11:41:36" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 11:43:00" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 11:44:24" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 11:45:36" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 11:47:36" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 11:48:00" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 11:55:24" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 11:56:48" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 11:57:12" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 12:00:12" level=error msg="Failed to get block number from timestamp" error="Post "https://goerli.prylabs.net": context deadline exceeded" prefix="rpc/validator"
time="2020-07-29 12:25:29" level=error msg="Failed to close stream" error="read tcp4 172.31.2.203:9000->117.102.178.66:13000: read: connection reset by peer" prefix=sync
time="2020-07-29 12:48:26" level=error msg="Could not verify attestation" error="signature did not verify" prefix=sync
time="2020-07-29 12:49:38" level=error msg="Could not verify attestation" error="signature did not verify" prefix=sync
time="2020-07-29 12:49:50" level=error msg="Could not verify attestation" error="signature did not verify" prefix=sync
time="2020-07-29 12:49:50" level=error msg="Could not verify attestation" error="signature did not verify" prefix=sync
Validator
time="2020-07-28 06:36:26" level=error msg="Could not request attestation to sign at slot" error="rpc error: code = InvalidArgument desc = invalid request: attestation slot 59604 not within attestation propagation range of 59571 to 59603 (current slot)" prefix=validator pubKey=0xb05912611a8a slot=59604
time="2020-07-28 06:36:30" level=error msg="Could not submit slot signature to beacon node: rpc error: code = Internal desc = Could not find attestation for slot and committee in pool" prefix=validator slot=59604
time="2020-07-28 06:36:38" level=error msg="Could not request attestation to sign at slot" error="rpc error: code = InvalidArgument desc = invalid request: attestation slot 59605 not within attestation propagation range of 59572 to 59604 (current slot)" prefix=validator pubKey=0xab96f762e34f slot=59605
time="2020-07-28 06:36:42" level=error msg="Could not submit slot signature to beacon node: rpc error: code = Internal desc = Could not find attestation for slot and committee in pool" prefix=validator slot=59605
time="2020-07-28 06:36:48" level=error msg="Failed to propose block" error="rpc error: code = Internal desc = Could not process beacon block: could not process block: could not process slot from the future, slot time 2020-07-28 06:36:46 +0000 UTC > current time 2020-07-28 06:36:42.97448128 +0000 UTC m=+716003.074494488" prefix=validator pubKey=0xab862426225a
time="2020-07-28 06:36:50" level=error msg="Could not request attestation to sign at slot" error="rpc error: code = InvalidArgument desc = invalid request: attestation slot 59606 not within attestation propagation range of 59573 to 59605 (current slot)" prefix=validator pubKey=0x90c2d8caa017 slot=59606
time="2020-07-28 06:36:50" level=error msg="Could not request attestation to sign at slot" error="rpc error: code = InvalidArgument desc = invalid request: attestation slot 59606 not within attestation propagation range of 59573 to 59605 (current slot)" prefix=validator pubKey=0x93fecf223672 slot=59606
time="2020-07-28 06:36:54" level=error msg="Could not submit slot signature to beacon node: rpc error: code = Internal desc = Could not find attestation for slot and committee in pool" prefix=validator slot=59606
time="2020-07-28 06:37:02" level=error msg="Could not request attestation to sign at slot" error="rpc error: code = InvalidArgument desc = invalid request: attestation slot 59607 not within attestation propagation range of 59574 to 59606 (current slot)" prefix=validator pubKey=0x92391f2f8f70 slot=59607
time="2020-07-28 06:37:06" level=error msg="Could not submit slot signature to beacon node: rpc error: code = Internal desc = Could not find attestation for slot and committee in pool" prefix=validator slot=59607
time="2020-07-28 06:37:12" level=error msg="Failed to propose block" error="rpc error: code = Internal desc = Could not process beacon block: could not process block: could not process slot from the future, slot time 2020-07-28 06:37:10 +0000 UTC > current time 2020-07-28 06:37:06.946614948 +0000 UTC m=+716027.046628108" prefix=validator pubKey=0xb8c30fe77f4c
time="2020-07-28 06:37:14" level=error msg="Could not request attestation to sign at slot" error="rpc error: code = InvalidArgument desc = invalid request: attestation slot 59608 not within attestation propagation range of 59575 to 59607 (current slot)" prefix=validator pubKey=0xa89b0f62f290 slot=59608
time="2020-07-28 06:37:18" level=error msg="Could not submit slot signature to beacon node: rpc error: code = Internal desc = Could not find attestation for slot and committee in pool" prefix=validator slot=59608
time="2020-07-28 06:37:38" level=error msg="Could not request attestation to sign at slot" error="rpc error: code = InvalidArgument desc = invalid request: attestation slot 59610 not within attestation propagation range of 59577 to 59609 (current slot)" prefix=validator pubKey=0x8a2076344a20 slot=59610
time="2020-07-28 06:37:42" level=error msg="Could not submit slot signature to beacon node: rpc error: code = Internal desc = Could not find attestation for slot and committee in pool" prefix=validator slot=59610
time="2020-07-28 06:38:00" level=error msg="Failed to propose block" error="rpc error: code = Internal desc = Could not process beacon block: could not process block: could not process slot from the future, slot time 2020-07-28 06:37:58 +0000 UTC > current time 2020-07-28 06:37:54.895913532 +0000 UTC m=+716074.995926635" prefix=validator pubKey=0x8116f9d43628
time="2020-07-28 06:38:26" level=error msg="Could not request attestation to sign at slot" error="rpc error: code = InvalidArgument desc = invalid request: attestation slot 59614 not within attestation propagation range of 59581 to 59613 (current slot)" prefix=validator pubKey=0xb637a6cce122 slot=59614
time="2020-07-28 06:38:30" level=error msg="Could not submit slot signature to beacon node: rpc error: code = Internal desc = Could not find attestation for slot and committee in pool" prefix=validator slot=59614
time="2020-07-28 06:38:38" level=error msg="Could not request attestation to sign at slot" error="rpc error: code = InvalidArgument desc = invalid request: attestation slot 59615 not within attestation propagation range of 59582 to 59614 (current slot)" prefix=validator pubKey=0xa0b3af7ad720 slot=59615
time="2020-07-28 06:38:38" level=error msg="Could not request attestation to sign at slot" error="rpc error: code = InvalidArgument desc = invalid request: attestation slot 59615 not within attestation propagation range of 59582 to 59614 (current slot)" prefix=validator pubKey=0x8ff351cec33c slot=59615
time="2020-07-28 06:38:38" level=error msg="Could not request attestation to sign at slot" error="rpc error: code = InvalidArgument desc = invalid request: attestation slot 59615 not within attestation propagation range of 59582 to 59614 (current slot)" prefix=validator pubKey=0x8496e00cb85c slot=59615
time="2020-07-28 06:38:42" level=error msg="Could not submit slot signature to beacon node: rpc error: code = Internal desc = Could not find attestation for slot and committee in pool" prefix=validator slot=59615
time="2020-07-28 06:38:50" level=error msg="Could not request attestation to sign at slot" error="rpc error: code = InvalidArgument desc = invalid request: attestation slot 59616 not within attestation propagation range of 59583 to 59615 (current slot)" prefix=validator pubKey=0xa89b0f62f290 slot=59616
time="2020-07-28 06:38:50" level=error msg="Could not request attestation to sign at slot" error="rpc error: code = InvalidArgument desc = invalid request: attestation slot 59616 not within attestation propagation range of 59583 to 59615 (current slot)" prefix=validator pubKey=0xa070a71012c8 slot=59616
time="2020-07-28 06:38:54" level=error msg="Could not submit slot signature to beacon node: rpc error: code = Internal desc = Could not find attestation for slot and committee in pool" prefix=validator slot=59616
time="2020-07-28 06:39:00" level=error msg="Failed to propose block" error="rpc error: code = Internal desc = Could not process beacon block: could not process block: could not process slot from the future, slot time 2020-07-28 06:38:58 +0000 UTC > current time 2020-07-28 06:38:54.890112797 +0000 UTC m=+716134.990125974" prefix=validator pubKey=0x920fb32fc5a7
time="2020-07-28 06:39:02" level=error msg="Could not request attestation to sign at slot" error="rpc error: code = InvalidArgument desc = invalid request: attestation slot 59617 not within attestation propagation range of 59584 to 59616 (current slot)" prefix=validator pubKey=0x920fb32fc5a7 slot=59617
time="2020-07-28 06:39:02" level=error msg="Could not request attestation to sign at slot" error="rpc error: code = InvalidArgument desc = invalid request: attestation slot 59617 not within attestation propagation range of 59584 to 59616 (current slot)" prefix=validator pubKey=0xaaa15caad703 slot=59617
time="2020-07-28 06:39:06" level=error msg="Could not submit slot signature to beacon node: rpc error: code = Internal desc = Could not find attestation for slot and committee in pool" prefix=validator slot=59617
time="2020-07-28 06:39:38" level=error msg="Could not request attestation to sign at slot" error="rpc error: code = InvalidArgument desc = invalid request: attestation slot 59620 not within attestation propagation range of 59587 to 59619 (current slot)" prefix=validator pubKey=0xb526e9a055a1 slot=59620
time="2020-07-28 06:39:42" level=error msg="Could not submit slot signature to beacon node: rpc error: code = Internal desc = Could not find attestation for slot and committee in pool" prefix=validator slot=59620
time="2020-07-28 06:39:50" level=error msg="Could not request attestation to sign at slot" error="rpc error: code = InvalidArgument desc = invalid request: attestation slot 59621 not within attestation propagation range of 59588 to 59620 (current slot)" prefix=validator pubKey=0x88c6c308f5d5 slot=59621
time="2020-07-28 06:39:50" level=error msg="Could not request attestation to sign at slot" error="rpc error: code = InvalidArgument desc = invalid request: attestation slot 59621 not within attestation propagation range of 59588 to 59620 (current slot)" prefix=validator pubKey=0xb637a6cce122 slot=59621
time="2020-07-28 06:39:54" level=error msg="Could not submit slot signature to beacon node: rpc error: code = Internal desc = Could not find attestation for slot and committee in pool" prefix=validator slot=59621
time="2020-07-28 06:40:12" level=error msg="Failed to propose block" error="rpc error: code = Internal desc = Could not process beacon block: could not process block: could not process slot from the future, slot time 2020-07-28 06:40:10 +0000 UTC > current time 2020-07-28 06:40:06.710014217 +0000 UTC m=+716206.810027394" prefix=validator pubKey=0x84dd17e4b9c8
time="2020-07-28 06:40:26" level=error msg="Could not request attestation to sign at slot" error="rpc error: code = InvalidArgument desc = invalid request: attestation slot 59624 not within attestation propagation range of 59591 to 59623 (current slot)" prefix=validator pubKey=0xa690f6af92cb slot=59624
time="2020-07-28 06:40:30" level=error msg="Could not submit slot signature to beacon node: rpc error: code = Internal desc = Could not find attestation for slot and committee in pool" prefix=validator slot=59624
time="2020-07-28 06:40:38" level=error msg="Could not request attestation to sign at slot" error="rpc error: code = InvalidArgument desc = invalid request: attestation slot 59625 not within attestation propagation range of 59592 to 59624 (current slot)" prefix=validator pubKey=0x8c618479377e slot=59625
time="2020-07-28 06:40:42" level=error msg="Could not submit slot signature to beacon node: rpc error: code = Internal desc = Could not find attestation for slot and committee in pool" prefix=validator slot=59625
time="2020-07-28 06:40:50" level=error msg="Could not request attestation to sign at slot" error="rpc error: code = InvalidArgument desc = invalid request: attestation slot 59626 not within attestation propagation range of 59593 to 59625 (current slot)" prefix=validator pubKey=0x8116f9d43628 slot=59626
time="2020-07-28 06:40:54" level=error msg="Could not submit slot signature to beacon node: rpc error: code = Internal desc = Could not find attestation for slot and committee in pool" prefix=validator slot=59626
time="2020-07-28 06:41:02" level=error msg="Could not request attestation to sign at slot" error="rpc error: code = InvalidArgument desc = invalid request: attestation slot 59627 not within attestation propagation range of 59594 to 59626 (current slot)" prefix=validator pubKey=0x8a2076344a20 slot=59627
time="2020-07-28 06:41:06" level=error msg="Could not submit slot signature to beacon node: rpc error: code = Internal desc = Could not find attestation for slot and committee in pool" prefix=validator slot=59627
time="2020-07-28 06:41:14" level=error msg="Could not request attestation to sign at slot" error="rpc error: code = InvalidArgument desc = invalid request: attestation slot 59628 not within attestation propagation range of 59595 to 59627 (current slot)" prefix=validator pubKey=0x8f8829801df1 slot=59628
time="2020-07-28 06:41:14" level=error msg="Could not request attestation to sign at slot" error="rpc error: code = InvalidArgument desc = invalid request: attestation slot 59628 not within attestation propagation range of 59595 to 59627 (current slot)" prefix=validator pubKey=0xab96f762e34f slot=59628
time="2020-07-28 06:41:18" level=error msg="Could not submit slot signature to beacon node: rpc error: code = Internal desc = Could not find attestation for slot and committee in pool" prefix=validator slot=59628
time="2020-07-28 06:41:38" level=error msg="Could not request attestation to sign at slot" error="rpc error: code = InvalidArgument desc = invalid request: attestation slot 59630 not within attestation propagation range of 59597 to 59629 (current slot)" prefix=validator pubKey=0x879bd7308de1 slot=59630
time="2020-07-28 06:41:42" level=error msg="Could not submit slot signature to beacon node: rpc error: code = Internal desc = Could not find attestation for slot and committee in pool" prefix=validator slot=59630
time="2020-07-28 06:41:50" level=error msg="Could not request attestation to sign at slot" error="rpc error: code = InvalidArgument desc = invalid request: attestation slot 59631 not within attestation propagation range of 59598 to 59630 (current slot)" prefix=validator pubKey=0xab862426225a slot=59631
time="2020-07-28 06:41:54" level=error msg="Could not submit slot signature to beacon node: rpc error: code = Internal desc = Could not find attestation for slot and committee in pool" prefix=validator slot=59631
time="2020-07-28 06:42:26" level=error msg="Could not request attestation to sign at slot" error="rpc error: code = InvalidArgument desc = invalid request: attestation slot 59634 not within attestation propagation range of 59601 to 59633 (current slot)" prefix=validator pubKey=0xad9db6ff6f77 slot=59634
time="2020-07-28 06:42:30" level=error msg="Could not submit slot signature to beacon node: rpc error: code = Internal desc = Could not find attestation for slot and committee in pool" prefix=validator slot=59634
time="2020-07-28 06:42:38" level=error msg="Could not request attestation to sign at slot" error="rpc error: code = InvalidArgument desc = invalid request: attestation slot 59635 not within attestation propagation range of 59602 to 59634 (current slot)" prefix=validator pubKey=0x90c2d8caa017 slot=59635
time="2020-07-28 06:42:38" level=error msg="Could not request attestation to sign at slot" error="rpc error: code = InvalidArgument desc = invalid request: attestation slot 59635 not within attestation propagation range of 59602 to 59634 (current slot)" prefix=validator pubKey=0x827a91c95bf7 slot=59635
time="2020-07-28 06:42:38" level=error msg="Could not request attestation to sign at slot" error="rpc error: code = InvalidArgument desc = invalid request: attestation slot 59635 not within attestation propagation range of 59602 to 59634 (current slot)" prefix=validator pubKey=0x93fecf223672 slot=59635
time="2020-07-28 06:42:42" level=error msg="Could not submit slot signature to beacon node: rpc error: code = Internal desc = Could not find attestation for slot and committee in pool" prefix=validator slot=59635
time="2020-07-28 06:42:48" level=error msg="Failed to propose block" error="rpc error: code = Internal desc = Could not process beacon block: could not process block: could not process slot from the future, slot time 2020-07-28 06:42:46 +0000 UTC > current time 2020-07-28 06:42:42.715815143 +0000 UTC m=+716362.815828289" prefix=validator pubKey=0xab862426225a
time="2020-07-28 06:42:50" level=error msg="Could not request attestation to sign at slot" error="rpc error: code = InvalidArgument desc = invalid request: attestation slot 59636 not within attestation propagation range of 59603 to 59635 (current slot)" prefix=validator pubKey=0xb8c30fe77f4c slot=59636
time="2020-07-28 06:42:54" level=error msg="Could not submit slot signature to beacon node: rpc error: code = Internal desc = Could not find attestation for slot and committee in pool" prefix=validator slot=59636
time="2020-07-28 06:43:00" level=error msg="Failed to propose block" error="rpc error: code = Internal desc = Could not process beacon block: could not process block: could not process slot from the future, slot time 2020-07-28 06:42:58 +0000 UTC > current time 2020-07-28 06:42:54.713747693 +0000 UTC m=+716374.813761079" prefix=validator pubKey=0xb05912611a8a
time="2020-07-28 06:43:14" level=error msg="Could not request attestation to sign at slot" error="rpc error: code = InvalidArgument desc = invalid request: attestation slot 59638 not within attestation propagation range of 59605 to 59637 (current slot)" prefix=validator pubKey=0x86e7ab2e6027 slot=59638
time="2020-07-28 06:43:14" level=error msg="Could not request attestation to sign at slot" error="rpc error: code = InvalidArgument desc = invalid request: attestation slot 59638 not within attestation propagation range of 59605 to 59637 (current slot)" prefix=validator pubKey=0x84dd17e4b9c8 slot=59638
time="2020-07-28 06:43:18" level=error msg="Could not submit slot signature to beacon node: rpc error: code = Internal desc = Could not find attestation for slot and committee in pool" prefix=validator slot=59638
The validator errors are interesting as we potentially have a internal clock mismatch, this will need more exploration on the root cause.
Looking through our code the root cause seems to be a skewed internal clock rather than any issue with goerli timeouts when proposing a block.
FYI, I tested the teku attacknet DoS issue on the prysm attacknet. It does not work as was working with teku.
Regarding clock skew, I would expect this to resolve itself after 1 to 2 hours after the attack stops. Perhaps we can resync the clock when we start to see a high volume of errors around clock related issues.
@protolambda, @djrtwo, waiting for your review 😁
@nisdas fixed it in this PR: https://github.com/prysmaticlabs/prysm/pull/6780
Great work @AlexSSD7! This qualifies for the $5k beta-0 prysm attacknet bounty. The clock skew is very interesting :)
We're a bit busy with preparations for the medalla testnet, but I'll circle back with you early next week to discuss payment and get you the leaderboard
Description
Prysm nodes are vulnerable to L4 DDoS attacks. The attacker can stop finality for any amount of time.
Attack scenario
There are 4 nodes on this network, and I've rented 4x10 DigitalOcean droplets and started the L4 DoS utility on each of them. It was enough to break the whole network P2P connectivity.
Impact
The nodes were unable to normally communicate with each other. As a result, new nodes were unable to sync, and validators submit their attestations.
Details
Attack period: 2109-2125 epochs
Attack requirements
Attack log
Proofs
How to mitigate?
Prysm nodes should temporarily ban IPs that flood their P2P port with the garbage traffic.
Note:
Lighthouse and teku are likely vulnerable too.