search

tronprotocol / java-tron

Java implementation of the Tron whitepaper
GNU Lesser General Public License v3.0
3.73k stars 1.41k forks source link

How to speed up node blocks sync? #5991

Closed thelotux closed 3 weeks ago

thelotux commented 1 month ago

Hi, I recently set up a fullnode. I attach the specifications of the hardware used, the configuration file(in .json) and filtered logs to show the statistics on the blocks. Regarding the startup I used the following command:

nohup java -Xmx80g -XX:+UseConcMarkSweepGC -jar FullNode.jar -c main_net_config.conf -d output-directory —es

Looking at the logs I noticed that the fetch operation of the blocks is very slow and has a delay ranging from 200ms to 800ms. I wanted to ask if this was normal and if there was a way to optimize its speed. Specifically I would like to use the node to make a subscription to the logs of a specific contract ( using Java-tron's Built-in Message Queue for Event Subscription ) but it's too slow.

System information

OS = Ubuntu 20.04.6 LTS CPU = Amd Ryzen 9 7950x3d Ram = DDR5 128gb Disk = 2 TB nvme ssd Bare metal Provider = Hetzner

tron.log main_net_config.json

abn2357 commented 1 month ago

you can try litefullnode

thelotux commented 1 month ago

I forgot to specify it but the node I set is the lite FullNode

tomatoishealthy commented 1 month ago 
PushBlock block number: 64972780, cost/txs: 25/319 false.

According to your log, I see that the block processing time is about 20-30ms. How do you calculate the time to be 200-800ms?

thelotux commented 1 month ago

I'm considering the log : Receive block/interval x from x fetch/delay 419/810ms specifically the delay. I then noticed that this coincides (considering the block processing and transmission delays) with the values ​​I obtain by monitoring the instant of receipt of transactions received via event subscription

jwrct commented 1 month ago

@jacopodemarco The two delay times you mentioned: the first is the block fetching delay, which is the actual network delay, not the delay in business processing; the second is the delay time of receiving blocks relative to block time, which includes block packaging time and network interaction time, as well as possible delays in the SR starting block production. From the logs you provided, your node did not appear to be lagging behind in blocks, so it is normal. If normal delays do not meet your business needs, we suggest increasing your node's network bandwidth and hardware, which should help improve your node's block synchronization speed.

thelotux commented 1 month ago

I've noticed that some peers allow me to achieve much lower delays than others. Can I somehow specify which peers to use and which to blacklist so as not to use them?

jwrct commented 1 month ago

@jacopodemarco Usually, the peer that broadcasts the block inventory first will request the block from that peer. This is usually the best choice. Currently, it does not support specifying peers to request blocks, nor does it support blacklist.

abn2357 commented 1 month ago

Currently, It is recommended to use specifications for a Fullnode as below

CPU：16 cores RAM：32G Bandwidth：100M SSD：2.5T+

In the recommended config and network condition, It will cost 300ms to sync one block, you can enhance your hardware and network to speed sync

abn2357 commented 1 month ago

@jacopodemarco The two delay times you mentioned: the first is the block fetching delay, which is the actual network delay, not the delay in business processing; the second is the delay time of receiving blocks relative to block time, which includes block packaging time and network interaction time, as well as possible delays in the SR starting block production. From the logs you provided, your node did not appear to be lagging behind in blocks, so it is normal. If normal delays do not meet your business needs, we suggest increasing your node's network bandwidth and hardware, which should help improve your node's block synchronization speed.

what does block time mean in "the second is the delay time of receiving blocks relative to block time"

jwrct commented 1 month ago

@jacopodemarco The two delay times you mentioned: the first is the block fetching delay, which is the actual network delay, not the delay in business processing; the second is the delay time of receiving blocks relative to block time, which includes block packaging time and network interaction time, as well as possible delays in the SR starting block production. From the logs you provided, your node did not appear to be lagging behind in blocks, so it is normal. If normal delays do not meet your business needs, we suggest increasing your node's network bandwidth and hardware, which should help improve your node's block synchronization speed.

what does block time mean in "the second is the delay time of receiving blocks relative to block time"

Block time referred here refers to the timestamp in the block header.

N7DTM commented 1 month ago

TRON FullNode Deployment Instructions

1. Use the Latest Config

Download the latest configuration file from the tronprotocol GitHub repository.

2. Download the Appropriate Snapshot

For db.engine = "LEVELDB", download the appropriate snapshot from the TRON Mainnet Snapshot Documentation:

wget -bqc http://34.143.247.77/backup20240923/FullNode_output-directory.tgz

3. Check Download Status

ps aux | grep wget

4. Extract Snapshot in Background

nohup tar zxvf FullNode_output-directory.tgz -C /root/output-directory/ > /root/tar.log &

(Optional) Download Multiple Snapshots Simultaneously for Faster Loading:

If you'd like to speed up the process, you can download from multiple servers simultaneously. This increases the chance of faster retrieval:

A. Download Multiple Snapshots Simultaneously

To increase the chance of a faster download:

wget -bqc http://34.143.247.77/backup20240923/FullNode_output-directory.tgz -O FullNode_output-directory-1.tgz
wget -bqc http://34.86.86.229/backup20240924/FullNode_output-directory.tgz -O FullNode_output-directory-2.tgz

B. Monitor Download Progress

After 5 minutes, check which snapshot is downloading faster:

du -sh FullNode_output-directory-1.tgz FullNode_output-directory-2.tgz

C. Terminate Slower Download

Stop the slower download and remove the file:

kill <ID> && rm FullNode_output-directory-1.tgz

D. Extract the Faster Download

Extract the faster snapshot:

nohup tar zxvf FullNode_output-directory-2.tgz -C /root/ > /root/tar.log &
vivian1912 commented 3 weeks ago

@thelotux May I ask if the above answer your question? If yes, this issue will be closed later and please feel free to re-open it if you still see the issue, thanks.