search

migalabs / armiarma

Armiarma is a Libp2p open-network crawler with a current focus on Ethereum's CL network
https://monitoreth.io
MIT License
36 stars 13 forks source link

Default value for persist-connevents breaks #61

Closed santi1234567 closed 10 months ago

santi1234567 commented 10 months ago

Description

When running the crawler with the default setup using docker, the following error appears:

eth_crawler_1  | 2023-08-30T16:08:20.869044418Z time="2023-08-30T16:08:20Z" level=error msg="error: invalid boolean value \"\" for -persist-connevents: parse error\n" module=ARMIARMA
eth_crawler_1  | 2023-08-30T16:09:21.446923654Z Armiarma_v2.0.0
eth_crawler_1  | 2023-08-30T16:09:21.447061051Z 
eth_crawler_1  | 2023-08-30T16:09:21.447535901Z Incorrect Usage: invalid boolean value "" for -persist-connevents: parse error

It seems to take "" as a value and not fallback to the default value (false).

cortze commented 10 months ago

Hey @santi1234567, thanks for reporting the issues. Unfortunately, I can't reproduce the error using the default configuration: 

$ docker-compose --env-file=".env_template" up db eth_crawler 
[+] Running 2/0
 ✔ Container armiarma-db-1           Created                                                                                                                                                                              
 ✔ Container armiarma-eth_crawler-1  Recreated             
armiarma-db-1           | 
armiarma-db-1           | PostgreSQL Database directory appears to contain a database; Skipping initialization
armiarma-db-1           | 
armiarma-db-1           | 
armiarma-db-1           | 2023-08-31 09:06:12.327 UTC [1] LOG:  starting PostgreSQL 14.1 on x86_64-pc-linux-musl, compiled by gcc (Alpine 10.3.1_git20211027) 10.3.1 20211027, 64-bit
armiarma-db-1           | 2023-08-31 09:06:12.328 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
armiarma-db-1           | 2023-08-31 09:06:12.328 UTC [1] LOG:  listening on IPv6 address "::", port 5432
armiarma-db-1           | 2023-08-31 09:06:12.330 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
armiarma-db-1           | 2023-08-31 09:06:12.334 UTC [21] LOG:  database system was shut down at 2023-08-29 13:15:47 UTC
armiarma-db-1           | 2023-08-31 09:06:12.340 UTC [1] LOG:  database system is ready to accept connections
armiarma-eth_crawler-1  | Armiarma_v2.0.0
armiarma-eth_crawler-1  | 
armiarma-eth_crawler-1  | 
armiarma-eth_crawler-1  | time="2023-08-31T09:06:22Z" level=info msg="Starting Ethereum Crawler..."
armiarma-eth_crawler-1  | time="2023-08-31T09:06:22Z" level=info msg="config for the Ethereum crawler" backup-interval=12h bootnodes="[enr:-KG4QOtcP9X1FbIMOe17QNMKqDxCpm14jcX5tiOE4_TyMrFqbmhPZHK_ZPG2Gxb1GE2xdtodOfx9-cgvNtxnRyHEmC0ghGV0aDKQ9aX9QgAAAAD__________4JpZIJ2NIJpcIQDE8KdiXNlY3AyNTZrMaEDhpehBDbZjM_L9ek699Y7vhUJ-eAdMyQW_Fil522Y0fODdGNwgiMog3VkcIIjKA enr:-KG4QL-eqFoHy0cI31THvtZjpYUu_Jdw_MO7skQRJxY1g5HTN1A0epPCU6vi0gLGUgrzpU-ygeMSS8ewVxDpKfYmxMMGhGV0aDKQtTA_KgAAAAD__________4JpZIJ2NIJpcIQ2_DUbiXNlY3AyNTZrMaED8GJ2vzUqgL6-KD1xalo1CsmY4X1HaDnyl6Y_WayCo9GDdGNwgiMog3VkcIIjKA enr:-Ku4QImhMc1z8yCiNJ1TyUxdcfNucje3BGwEHzodEZUan8PherEo4sF7pPHPSIB1NNuSg5fZy7qFsjmUKs2ea1Whi0EBh2F0dG5ldHOIAAAAAAAAAACEZXRoMpD1pf1CAAAAAP__________gmlkgnY0gmlwhBLf22SJc2VjcDI1NmsxoQOVphkDqal4QzPMksc5wnpuC3gvSC8AfbFOnZY_On34wIN1ZHCCIyg enr:-Ku4QP2xDnEtUXIjzJ_DhlCRN9SN99RYQPJL92TMlSv7U5C1YnYLjwOQHgZIUXw6c-BvRg2Yc2QsZxxoS_pPRVe0yK8Bh2F0dG5ldHOIAAAAAAAAAACEZXRoMpD1pf1CAAAAAP__________gmlkgnY0gmlwhBLf22SJc2VjcDI1NmsxoQMeFF5GrS7UZpAH2Ly84aLK-TyvH-dRo0JM1i8yygH50YN1ZHCCJxA enr:-Ku4QPp9z1W4tAO8Ber_NQierYaOStqhDqQdOPY3bB3jDgkjcbk6YrEnVYIiCBbTxuar3CzS528d2iE7TdJsrL-dEKoBh2F0dG5ldHOIAAAAAAAAAACEZXRoMpD1pf1CAAAAAP__________gmlkgnY0gmlwhBLf22SJc2VjcDI1NmsxoQMw5fqqkw2hHC4F5HZZDPsNmPdB1Gi8JPQK7pRc9XHh-oN1ZHCCKvg enr:-Jq4QItoFUuug_n_qbYbU0OY04-np2wT8rUCauOOXNi0H3BWbDj-zbfZb7otA7jZ6flbBpx1LNZK2TDebZ9dEKx84LYBhGV0aDKQtTA_KgEAAAD__________4JpZIJ2NIJpcISsaa0ZiXNlY3AyNTZrMaEDHAD2JKYevx89W0CcFJFiskdcEzkH_Wdv9iW42qLK79ODdWRwgiMo enr:-Jq4QN_YBsUOqQsty1OGvYv48PMaiEt1AzGD1NkYQHaxZoTyVGqMYXg0K9c0LPNWC9pkXmggApp8nygYLsQwScwAgfgBhGV0aDKQtTA_KgEAAAD__________4JpZIJ2NIJpcISLosQxiXNlY3AyNTZrMaEDBJj7_dLFACaxBfaI8KZTh_SSJUjhyAyfshimvSqo22WDdWRwgiMo enr:-Ku4QHqVeJ8PPICcWk1vSn_XcSkjOkNiTg6Fmii5j6vUQgvzMc9L1goFnLKgXqBJspJjIsB91LTOleFmyWWrFVATGngBh2F0dG5ldHOIAAAAAAAAAACEZXRoMpC1MD8qAAAAAP__________gmlkgnY0gmlwhAMRHkWJc2VjcDI1NmsxoQKLVXFOhp2uX6jeT0DvvDpPcU8FWMjQdR4wMuORMhpX24N1ZHCCIyg enr:-Ku4QG-2_Md3sZIAUebGYT6g0SMskIml77l6yR-M_JXc-UdNHCmHQeOiMLbylPejyJsdAPsTHJyjJB2sYGDLe0dn8uYBh2F0dG5ldHOIAAAAAAAAAACEZXRoMpC1MD8qAAAAAP__________gmlkgnY0gmlwhBLY-NyJc2VjcDI1NmsxoQORcM6e19T1T9gi7jxEZjk_sjVLGFscUNqAY9obgZaxbIN1ZHCCIyg enr:-Ku4QPn5eVhcoF1opaFEvg1b6JNFD2rqVkHQ8HApOKK61OIcIXD127bKWgAtbwI7pnxx6cDyk_nI88TrZKQaGMZj0q0Bh2F0dG5ldHOIAAAAAAAAAACEZXRoMpC1MD8qAAAAAP__________gmlkgnY0gmlwhDayLMaJc2VjcDI1NmsxoQK2sBOLGcUb4AwuYzFuAVCaNHA-dy24UuEKkeFNgCVCsIN1ZHCCIyg enr:-Ku4QEWzdnVtXc2Q0ZVigfCGggOVB2Vc1ZCPEc6j21NIFLODSJbvNaef1g4PxhPwl_3kax86YPheFUSLXPRs98vvYsoBh2F0dG5ldHOIAAAAAAAAAACEZXRoMpC1MD8qAAAAAP__________gmlkgnY0gmlwhDZBrP2Jc2VjcDI1NmsxoQM6jr8Rb1ktLEsVcKAPa08wCsKUmvoQ8khiOl_SLozf9IN1ZHCCIyg enr:-LK4QA8FfhaAjlb_BXsXxSfiysR7R52Nhi9JBt4F8SPssu8hdE1BXQQEtVDC3qStCW60LSO7hEsVHv5zm8_6Vnjhcn0Bh2F0dG5ldHOIAAAAAAAAAACEZXRoMpC1MD8qAAAAAP__________gmlkgnY0gmlwhAN4aBKJc2VjcDI1NmsxoQJerDhsJ-KxZ8sHySMOCmTO6sHM3iCFQ6VMvLTe948MyYN0Y3CCI4yDdWRwgiOM enr:-LK4QKWrXTpV9T78hNG6s8AM6IO4XH9kFT91uZtFg1GcsJ6dKovDOr1jtAAFPnS2lvNltkOGA9k29BUN7lFh_sjuc9QBh2F0dG5ldHOIAAAAAAAAAACEZXRoMpC1MD8qAAAAAP__________gmlkgnY0gmlwhANAdd-Jc2VjcDI1NmsxoQLQa6ai7y9PMN5hpLe5HmiJSlYzMuzP7ZhwRiwHvqNXdoN0Y3CCI4yDdWRwgiOM]" cl-endpoint= fork-digest=0x4a26c58b gossip-topics="[beacon_block]" ip=0.0.0.0 log-level=info persist-connevents=false persist-msgs=false port=9020 priv-key= psql="postgres://user:password@db:5432/armiarmadb" subnets="[1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64]" user-agent="Armiarma Crawler" val-pubkeys=0
armiarma-eth_crawler-1  | time="2023-08-31T09:06:22Z" level=info msg="Creating Local Node"
armiarma-eth_crawler-1  | time="2023-08-31T09:06:22Z" level=info msg="init active_peers table"
armiarma-eth_crawler-1  | time="2023-08-31T09:06:22Z" level=info msg="init eth_attestations table in psql-db"
armiarma-eth_crawler-1  | time="2023-08-31T09:06:22Z" level=info msg="init eth_blocks table in psql-db"
armiarma-eth_crawler-1  | time="2023-08-31T09:06:22Z" level=info msg="libp2p successfully generated" maddrs=/ip4/0.0.0.0/tcp/9020 peerID=16Uiu2HAkwdA7dRQhWrvkkZV7YCe9Tucc19n2ijnjUyKbw9YxYCr6
armiarma-eth_crawler-1  | time="2023-08-31T09:06:22Z" level=info msg="launching discovery5 at fork 0x4a26c58b"
armiarma-eth_crawler-1  | time="2023-08-31T09:06:22Z" level=info msg="configuring crawler with peering strategy: %spruning"
armiarma-eth_crawler-1  | time="2023-08-31T09:06:22Z" level=info msg="IP locator routine started"
armiarma-eth_crawler-1  | time="2023-08-31T09:06:22Z" level=info msg="Started serving Beacon Status"
armiarma-eth_crawler-1  | time="2023-08-31T09:06:22Z" level=info msg="starting the peering service"
armiarma-eth_crawler-1  | time="2023-08-31T09:06:22Z" level=info msg="Started serving Beacon Metadata"
armiarma-eth_crawler-1  | time="2023-08-31T09:06:22Z" level=info msg="Started serving ping"
armiarma-eth_crawler-1  | time="2023-08-31T09:06:22Z" level=info msg="initializing exporter client_distribution"
armiarma-eth_crawler-1  | time="2023-08-31T09:06:22Z" level=info msg="initializing exporter client_version_distribution"
armiarma-eth_crawler-1  | time="2023-08-31T09:06:22Z" level=info msg="initializing exporter geographical_distribution"
armiarma-eth_crawler-1  | time="2023-08-31T09:06:22Z" level=info msg="initializing exporter geographical_distribution"
armiarma-eth_crawler-1  | time="2023-08-31T09:06:22Z" level=info msg="initializing exporter deprecated_nodes"
armiarma-eth_crawler-1  | time="2023-08-31T09:06:22Z" level=info msg="initializing exporter os_distribution"
armiarma-eth_crawler-1  | time="2023-08-31T09:06:22Z" level=info msg="initializing exporter arch_distribution"
armiarma-eth_crawler-1  | time="2023-08-31T09:06:22Z" level=info msg="initializing exporter hosted_peer_distribution"
armiarma-eth_crawler-1  | time="2023-08-31T09:06:22Z" level=info msg="initializing exporter rtt_distribution"
armiarma-eth_crawler-1  | time="2023-08-31T09:06:22Z" level=info msg="initializing exporter ip_distribution"
armiarma-eth_crawler-1  | time="2023-08-31T09:06:22Z" level=info msg="initializing exporter pruned_error_distribution"
armiarma-eth_crawler-1  | time="2023-08-31T09:06:22Z" level=info msg="initializing exporter iteration_attempts_by_category_distribution"
armiarma-eth_crawler-1  | time="2023-08-31T09:06:22Z" level=info msg="initializing exporter peers_attempted_last_iteration"
armiarma-eth_crawler-1  | time="2023-08-31T09:06:22Z" level=info msg="initializing exporter peerstore_iteration_time_secs"
armiarma-eth_crawler-1  | time="2023-08-31T09:06:22Z" level=info msg="initializing exporter conn_error_distribution"
armiarma-eth_crawler-1  | time="2023-08-31T09:06:22Z" level=info msg="initializing exporter total_conn_error_distribution"
armiarma-eth_crawler-1  | time="2023-08-31T09:06:22Z" level=info msg="initializing exporter node_per_fork_distribution"
armiarma-eth_crawler-1  | time="2023-08-31T09:06:22Z" level=info msg="initializing exporter attnets_distribution"
armiarma-eth_crawler-1  | time="2023-08-31T09:06:22Z" level=info msg="initializing exporter connected_peers"
armiarma-eth_crawler-1  | time="2023-08-31T09:06:22Z" level=info msg="initializing exporter suported protocols"
armiarma-eth_crawler-1  | time="2023-08-31T09:06:22Z" level=info msg="initializing exporter peers_per_topic"
armiarma-eth_crawler-1  | time="2023-08-31T09:06:22Z" level=info msg="initializing exporter local_head_slot"

Can you post the .env that recreated the issue? I'd expect that the error would come from a variable assigment to an empty string, but the default template does have an "false" string for that flag

santi1234567 commented 10 months ago

Hmm, I think that the issue then is that I assumed that if no .env file was present, this would be the "default" configuration. With the configs from the template, it works alright. I missunderstood the README in this case, my bad!.

cortze commented 10 months ago

closed with #63