search

EOSIO / history-tools

EOSIO History Tools
https://eosio.github.io/history-tools/
MIT License
65 stars 45 forks source link

catch_and_close Stream overrun #103

Open cppfuns opened 3 years ago

cppfuns commented 3 years ago

Run thefill-pg command for the first time as follows

./fill-pg --fill-connect-to 127.0.0.1:8080 --fpg-create

Check PostgreSQL at this time, the corresponding table has been created image But after checking the terminal returns an error The data in the table is not written

info  2020-12-24T06:45:17.195 fill-pg   main.cpp:98                   main                 ] fill-pg version v0.3.0-alpha-26-gf415e46
info  2020-12-24T06:45:17.195 fill-pg   main.cpp:99                   main                 ] fill-pg using configuration file /root/.local/share/eosio/fill-pg/config/config.ini
info  2020-12-24T06:45:17.195 fill-pg   main.cpp:100                  main                 ] fill-pg data directory is /root/.local/share/eosio/fill-pg/data
info  2020-12-24T06:45:17.195 fill-pg   fill_pg_plugin.cpp:98         fpg_session          ] connect to postgresql
info  2020-12-24T06:45:17.927 fill-pg   state_history_connecti:63     connect              ] connect to 127.0.0.1:8080
error 2020-12-24T06:45:17.928 fill-pg   state_history_connecti:168    catch_and_close      ] Stream overrun
info  2020-12-24T06:45:17.928 fill-pg   state_history_connecti:193    close                ] closing state-history socket

node version:v2.0.7 os: ubuntu 18.04

Related parameters in config.ini

state-history-dir = "state-history"
trace-history = true
chain-state-history = true
plugin = eosio::state_history_plugin
state-history-endpoint = 0.0.0.0:8080
cppfuns commented 3 years ago

@tbfleming @larryk85

jgiszczak commented 3 years ago

@cppfuns Please try again with the current develop branch of history-tools. We have been resolving compatibility issues there.

garden2weini commented 3 years ago

@cppfuns Please try again with the current develop branch of history-tools. We have been resolving compatibility issues there.

I clone develop branch of history-tools, and buld a docker image using ubuntu-18.04.dockerfile. But the same error still occur.

root@iZj6c0lvzs7170x7omf10oZ:~/history-tools/build# ./fill-pg --fill-connect-to 127.0.0.1:8080
info  2021-03-12T09:23:05.178 fill-pg   main.cpp:98                   main                 ] fill-pg version Unknown
info  2021-03-12T09:23:05.178 fill-pg   main.cpp:99                   main                 ] fill-pg using configuration file /root/.local/share/eosio/fill-pg/config/config.ini
info  2021-03-12T09:23:05.179 fill-pg   main.cpp:100                  main                 ] fill-pg data directory is /root/.local/share/eosio/fill-pg/data
info  2021-03-12T09:23:05.180 fill-pg   fill_pg_plugin.cpp:98         fpg_session          ] connect to postgresql
info  2021-03-12T09:23:05.203 fill-pg   state_history_connecti:63     connect              ] connect to 127.0.0.1:8080
error 2021-03-12T09:23:05.214 fill-pg   state_history_connecti:168    catch_and_close      ] Stream overrun
info  2021-03-12T09:23:05.215 fill-pg   state_history_connecti:193    close                ] closing state-history socket
elmato commented 3 years ago

Having the same problem with:

garden2weini commented 3 years ago

I pull eos and hitstory-tools from dockerhub, then can run history tools.
docker pull eosio/eosio:release_2.1.x docker pull eosio/history-tools:c763c194d5275de6d10175a9fc02342ea57560d8

elmato commented 3 years ago

@elmato I'd like to test my tiny fix in a various environments, so try one more time pls with patch from #104

Having the same problem with:

  • ship from eosio_2.1.0-rc3-ubuntu-18.04_amd64.deb
  • history-tools at c763c19

Thanks @633kh4ck, I tried it yesterday but with no luck, im still getting the same "Stream overrun" error as soon as fill-pg starts

elmato commented 3 years ago

@elmato I'd like to test my tiny fix in a various environments, so try one more time pls with patch from #104

Having the same problem with:

  • ship from eosio_2.1.0-rc3-ubuntu-18.04_amd64.deb
  • history-tools at c763c19

Thanks @633kh4ck, I tried it yesterday but with no luck, im still getting the same "Stream overrun" error as soon as fill-pg starts

Just for clarification: you need to build fill-pg (a-ka history-tools) from my fork (at 633kh4ck@7c251e4) because patch (#104) hasn't been merged yet

Building fill-pg from your branch and running it against ship from eosio_2.1.0-rc3-ubuntu-18.04_amd64.deb I'm getting a different error.

Bad variant index

➜  build git:(fix/safe-deltas-unpack) nodeos --version
v2.1.0-rc3
➜  build git:(fix/safe-deltas-unpack) ./fill-pg --fpg-drop --fpg-create --fill-connect-to=127.0.0.1:5432
info  2021-03-15T19:24:22.935 fill-pg   main.cpp:98                   main                 ] fill-pg version v0.3.0-alpha-27-g7c251e4
info  2021-03-15T19:24:22.935 fill-pg   main.cpp:99                   main                 ] fill-pg using configuration file /home/me/.local/share/eosio/fill-pg/config/config.ini
info  2021-03-15T19:24:22.935 fill-pg   main.cpp:100                  main                 ] fill-pg data directory is /home/me/.local/share/eosio/fill-pg/data
info  2021-03-15T19:24:22.935 fill-pg   fill_pg_plugin.cpp:98         fpg_session          ] connect to postgresql
info  2021-03-15T19:24:22.947 fill-pg   fill_pg_plugin.cpp:105        start                ] drop schema "chain"
info  2021-03-15T19:24:22.947 fill-pg   state_history_connecti:63     connect              ] connect to 127.0.0.1:5432
info  2021-03-15T19:24:22.950 fill-pg   fill_pg_plugin.cpp:230        create_tables        ] create schema "chain"
error 2021-03-15T19:24:23.073 fill-pg   state_history_connecti:168    catch_and_close      ] Bad variant index
info  2021-03-15T19:24:23.073 fill-pg   state_history_connecti:193    close                ] closing state-history socket
garden2weini commented 3 years ago

@garden2weini You just pulled the docker images (the commands you specified) - it has nothing to do with their actual usage. Have you tested them? Can you provide us with real details and logs pls?

I pull eos and hitstory-tools from dockerhub, then can run history tools. docker pull eosio/eosio:release_2.1.x docker pull eosio/history-tools:c763c194d5275de6d10175a9fc02342ea57560d8

you can ref https://github.com/garden2weini/eos-state-history-client/blob/master/History-Tools.md

EOS logs

info  2021-03-16T03:32:59.857 nodeos    state_history_plugin.c:237    send_update          ] pushing result {"head":{"block_num":165524202},"last_irreversible":{"block_num":165523869},"this_block":{"block_num":165208036}} to send queue
info  2021-03-16T03:32:59.963 nodeos    state_history_plugin.c:237    send_update          ] pushing result {"head":{"block_num":165524202},"last_irreversible":{"block_num":165523869},"this_block":{"block_num":165208037}} to send queue
info  2021-03-16T03:33:00.039 nodeos    state_history_plugin.c:237    send_update          ] pushing result {"head":{"block_num":165524202},"last_irreversible":{"block_num":165523869},"this_block":{"block_num":165208038}} to send queue
info  2021-03-16T03:33:00.183 nodeos    state_history_plugin.c:237    send_update          ] pushing result {"head":{"block_num":165524202},"last_irreversible":{"block_num":165523869},"this_block":{"block_num":165208039}} to send queue
info  2021-03-16T03:33:00.396 nodeos    state_history_plugin.c:237    send_update          ] pushing result {"head":{"block_num":165524202},"last_irreversible":{"block_num":165523869},"this_block":{"block_num":165208040}} to send queue
info  2021-03-16T03:33:00.524 nodeos    state_history_plugin.c:237    send_update          ] pushing result {"head":{"block_num":165524202},"last_irreversible":{"block_num":165523869},"this_block":{"block_num":165208041}} to send queue
info  2021-03-16T03:33:00.742 nodeos    state_history_plugin.c:237    send_update          ] pushing result {"head":{"block_num":165524202},"last_irreversible":{"block_num":165523869},"this_block":{"block_num":165208042}} to send queue
info  2021-03-16T03:33:01.155 nodeos    state_history_plugin.c:237    send_update          ] pushing result {"head":{"block_num":165524204},"last_irreversible":{"block_num":165523869},"this_block":{"block_num":165208043}} to send queue
info  2021-03-16T03:33:03.192 nodeos    state_history_plugin.c:237    send_update          ] pushing result {"head":{"block_num":165524207},"last_irreversible":{"block_num":165523881},"this_block":{"block_num":165208044}} to send queue
info  2021-03-16T03:33:24.314 nodeos    state_history_plugin.c:237    send_update          ] pushing result {"head":{"block_num":165524253},"last_irreversible":{"block_num":165523917},"this_block":{"block_num":165208045}} to send queue
info  2021-03-16T03:33:26.955 net-18    net_plugin.cpp:3425           connection_monitor   ] p2p client connections: 0/3, peer connections: 12/8

History Tools logs

info  2021-03-16T02:35:19.328 fill-pg   main.cpp:98                   main                 ] fill-pg version Unknown
info  2021-03-16T02:35:19.329 fill-pg   main.cpp:99                   main                 ] fill-pg using configuration file /root/.local/share/eosio/fill-pg/config/config.ini
info  2021-03-16T02:35:19.330 fill-pg   main.cpp:100                  main                 ] fill-pg data directory is /root/.local/share/eosio/fill-pg/data
info  2021-03-16T02:35:19.332 fill-pg   fill_pg_plugin.cpp:98         fpg_session          ] connect to postgresql
info  2021-03-16T02:35:19.379 fill-pg   state_history_connecti:63     connect              ] connect to 127.0.0.1:8080
info  2021-03-16T02:36:00.478 fill-pg   fill_pg_plugin.cpp:512        received             ] switch forks at block 165206199
info  2021-03-16T02:36:00.478 fill-pg   fill_pg_plugin.cpp:521        received             ] block 165206199
info  2021-03-16T02:43:19.222 fill-pg   fill_pg_plugin.cpp:646        close_streams        ] block 165206200 - 165206399
info  2021-03-16T02:47:42.615 fill-pg   fill_pg_plugin.cpp:646        close_streams        ] block 165206400 - 165206599
info  2021-03-16T02:48:52.795 fill-pg   fill_pg_plugin.cpp:646        close_streams        ] block 165206600 - 165206799
info  2021-03-16T02:52:15.600 fill-pg   fill_pg_plugin.cpp:646        close_streams        ] block 165206800 - 165206999
info  2021-03-16T02:53:00.890 fill-pg   fill_pg_plugin.cpp:646        close_streams        ] block 165207000 - 165207199
info  2021-03-16T02:54:06.232 fill-pg   fill_pg_plugin.cpp:646        close_streams        ] block 165207200 - 165207399
info  2021-03-16T03:10:22.263 fill-pg   fill_pg_plugin.cpp:646        close_streams        ] block 165207400 - 165207599
info  2021-03-16T03:23:59.790 fill-pg   fill_pg_plugin.cpp:646        close_streams        ] block 165207600 - 165207799
info  2021-03-16T03:29:18.873 fill-pg   fill_pg_plugin.cpp:646        close_streams        ] block 165207800 - 165207999
pocrd commented 2 years ago

How to fix this issue in latest version?