The current version of the crawler is stable and works fine. However, the DB usage tends to increase with the new schema to track connections events. As the following table shows, the biggest part of the DB comes from the conn_events table, thus, this PR aims to allow users to define whether they want to keep track of these events or not
table
pretty size
bytes
"peer_info"
"64 MB"
67182592
"conn_events"
"13 GB"
13862699008
"ips"
"8352 kB"
8552448
"active_peers"
"8192 bytes"
8192
"eth_nodes"
"58 MB"
60481536
"eth_status"
"5872 kB"
6012928
"eth_attestations"
"0 bytes"
0
"eth_blocks"
"0 bytes"
0
Updates
[x] Add a new flag to choose whether we want to keep track of the conn_events or not
[x] Extend the DBClient to support DBOptions -> makes the interface cleaner
[x] Clean up the docker-compose file adding the dependency between the eth_crawler and the db
Description
The current version of the crawler is stable and works fine. However, the DB usage tends to increase with the new schema to track connections events. As the following table shows, the biggest part of the DB comes from the
conn_eventstable, thus, this PR aims to allow users to define whether they want to keep track of these events or notUpdates
DBOptions-> makes the interface cleanerdocker-composefile adding the dependency between theeth_crawlerand thedb