ldmberman
commented
3 years ago Hi @vird, you need to compile via ./rebar3 as debug tar to start a new weave, but I'd suggest trying out our Test Weave SDK instead.
Closed vird closed 3 years ago
ldmberman
commented
3 years ago Hi @vird, you need to compile via ./rebar3 as debug tar to start a new weave, but I'd suggest trying out our Test Weave SDK instead.
vird
commented
3 years ago ./rebar3 as debug tar
...
===> tarball /data/_arweave/arweave.testnet/_build/debug/rel/arweave/arweave-2.4.0.0.tar.gz successfully created!./_build/debug/rel/arweave/bin/arweave init
2021-04-13 10:10:27.222913
args: format: label: 2021-04-13 10:10:27.223756 crash_report 2021-04-13 10:10:27.224086 supervisor_report 2021-04-13 10:10:27.225647 supervisor_report 2021-0
4-13 10:10:27.226544 crash_report 2021-04-13 10:10:27.227745 std_info Kernel pid terminated (application_controller) ({application_start_failure,kernel,{{shutdown
,{failed_to_start_child,net_sup,{shutdown,{failed_to_start_child,net_kernel,{'EXIT',nodistribution}}}}},{ker
Crash dump is being written to: erl_crash.dump...done arweave is an auxiliary script, you should use ./_build/debug/rel/arweave/bin/start.
vird
commented
3 years ago Yeah, I was not attentive, but anyway ./_build/debug/rel/arweave/bin/start init
Launching Erlang Virtual Machine...
Exec: /usr/lib/erlang/erts-10.3.5.13/bin/erlexec -noshell -noinput +Bd -boot /data/_arweave/arweave.testnet/_build/debug/rel/arweave/releases/2.4.0.0/arweave -mode embedded -boot_var ERTS_LIB_DIR /usr/lib/erlang/lib -config /data/_arweave/arweave.testnet/_build/debug/rel/arweave/releases/2.4.0.0/sys.config -args_file /data/_arweave/arweave.testnet/_build/debug/rel/arweave/releases/2.4.0.0/vm.args -- foreground +Bi -run ar main init
Root: /data/_arweave/arweave.testnet/_build/debug/rel/arweave
/data/_arweave/arweave.testnet/_build/debug/rel/arweave
=INFO REPORT==== 13-Apr-2021::10:20:33.441658 ===
alarm_handler: {set,{system_memory_high_watermark,[]}}
=INFO REPORT==== 13-Apr-2021::10:20:33.442909 ===
alarm_handler: {set,{{disk_almost_full,"/snap/core/10908"},[]}}
=INFO REPORT==== 13-Apr-2021::10:20:33.443071 ===
alarm_handler: {set,{{disk_almost_full,"/snap/core/10958"},[]}}
2021-04-13T10:20:35.391826+00:00 [error] supervisor.erl:357 supervisor: {local,ar_node_sup}, errorContext: start_error, reason: {{case_clause,false},[{ar_block,hash_wallet_list,3,[...]},{ar_weave,init,4,...},{ar_node_worker,init,...},{gen_server,...}|...]}, offender: [{pid,undefined},{id,ar_node_worker},{mfargs,{ar_node_worker,start_link,[]}},{restart_type,permanent},{shutdown,5000},{child_type,worker}]
2021-04-13T10:20:35.392813+00:00 [error] supervisor.erl:357 supervisor: {local,ar_sup}, errorContext: start_error, reason: {shutdown,{failed_to_start_child,ar_node_worker,{{case_clause,false},[{ar_block,hash_wallet_list,...},{ar_weave,...}|...]}}}, offender: [{pid,undefined},{id,ar_node_sup},{mfargs,{ar_node_sup,start_link,[]}},{restart_type,permanent},{shutdown,5000},{child_type,supervisor}]
2021-04-13T10:20:35.393003+00:00 [error] proc_lib.erl:508 crasher: initial call: ar_node_worker:init/1, pid: <0.788.0>, registered_name: [], error: {{case_clause,false},[{ar_block,hash_wallet_list,3,[{file,"/data/_arweave/arweave.testnet/apps/arweave/src/ar_block.erl"},{line,36}]},{ar_weave,init,4,[{file,"/data/_arweave/arweave.testnet/apps/arweave/src/ar_weave.erl"},{line,25}]},{ar_node_worker,init,1,[{file,"/data/_arweave/arweave.testnet/apps/arweave/src/ar_node_worker.erl"},{line,70}]},{gen_server,in...
2021-04-13T10:20:35.403719+00:00 [error] proc_lib.erl:508 crasher: initial call: application_master:init/4, pid: <0.742.0>, registered_name: [], exit: {{{shutdown,{failed_to_start_child,ar_node_sup,{shutdown,{failed_to_start_child,ar_node_worker,{{case_clause,false},[{...}|...]}}}}},{ar,start,[normal,[]]}},[{application_master,init,4,[{file,"application_master.erl"},{line,138}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,249}]}]}, ancestors: [<0.741.0>], message_queue_len: 1, messages: [{'...
{"init terminating in do_boot",{{badmatch,{error,{arweave,{{shutdown,{failed_to_start_child,ar_node_sup,{shutdown,{failed_to_start_child,ar_node_worker,{{case_clause,false},[{ar_block,hash_wallet_list,3,[{file,"/data/_arweave/arweave.testnet/apps/arweave/src/ar_block.erl"},{line,36}]},{ar_weave,init,4,[{file,"/data/_arweave/arweave.testnet/apps/arweave/src/ar_weave.erl"},{line,25}]},{ar_node_worker,init,1,[{file,"/data/_arweave/arweave.testnet/apps/arweave/src/ar_node_worker.erl"},{line,70}]},{gen_server,init_it,2,[{file,"gen_server.erl"},{line,374}]},{gen_server,init_it,6,[{file,"gen_server.erl"},{line,342}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,249}]}]}}}}},{ar,start,[normal,[]]}}}}},[{ar,start,1,[{file,"/data/_arweave/arweave.testnet/apps/arweave/src/ar.erl"},{line,327}]},{init,start_em,1,[]},{init,do_boot,3,[]}]}}
Arweave Heartbeat: The Arweave server has terminated. It will restart in 15 seconds.
Arweave Heartbeat: If you would like to avoid this, press control+c to kill the server. Hm, could you remove the _build folder, rebuild and retry?
vird
commented
3 years ago Same errors. Note. I use feature/testnet branch. Is that correct?
ldmberman
commented
3 years ago Ah, I thought you use the master branch, sorry. I changed the build targets in feature/testnet - build it by ./rebar3 as testnet.
vird
commented
3 years ago ./rebar3 as testnet seems not working for me on feature/testnet, but on master
./rebar3 as debug tar
./_build/debug/rel/arweave/bin/start init port 2984
seems works
There are some log errors with
./_build/debug/rel/arweave/bin/start init mine mining_addr w6PJGgRIk_KE0t6xEKeam3vDamx_f0hBwfCk8wPmR-Q max_miners 1 port 2984
Initialising RandomX dataset for fast hashing. Swap height: 0, Key: <<"QXJ3ZWF2ZSBHZW5lc2lzIFJhbmRvbVggS2V5">>. The process may take several minutes.
RandomX dataset initialisation for swap height 0 complete.
2021-04-13T14:00:35.227536+00:00 [error] gen_server.erl:888 ** Generic server ar_header_sync terminating , ** Last message in was {'$gen_cast',{join,[{<<193,173,112,111,150,115,...
>>,12364,<<"?M%ƪ"...>>}],[{block,<<>>,...}]}}, ** When Server state == #{block_index => [{<<""...>>,12364,...}],cleanup_started => false,db => #Ref<0.1217530001.50724938.164857>,..
.}, ** Reason for termination == , ** {bad_return_value,last_stored_block_index_has_no_intersection_with_the_new_one}
2021-04-13T14:00:35.229221+00:00 [error] proc_lib.erl:508 crasher: initial call: ar_header_sync:init/1, pid: <0.777.0>, registered_name: ar_header_sync, exit: {{bad_return_value,la
st_stored_block_index_has_no_intersection_with_the_new_one},[{gen_server,handle_common_reply,8,[{file,"gen_server.erl"},{line,755}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.er
l"},{line,249}]}]}, ancestors: [ar_sup,<0.745.0>], message_queue_len: 0, messages: [], links: [<0.755.0>], dictionary: [], trap_exit: true, status: ...
2021-04-13T14:00:35.231452+00:00 [error] supervisor.erl:701 supervisor: {local,ar_sup}, errorContext: child_terminated, reason: {bad_return_value,last_stored_block_index_has_no_int
ersection_with_the_new_one}, offender: [{pid,<0.777.0>},{id,ar_header_sync},{mfargs,{ar_header_sync,start_link,[]}},{restart_type,permanent},{shutdown,5000},{child_type,worker}]
2021-04-13T14:00:35.232551+00:00 [error] gen_server.erl:888 ** Generic server ar_data_sync terminating , ** Last message in was {'$gen_cast',{join,[{<<193,173,112,111,150,115,159,2
37,182,...>>,12364,<<"?M%ƪQhs...>>}]}}, ** When Server state == {sync_data_state,{1,{{12364,0},nil,nil}},#{},[{<<""...>>,12364,...}],12364,{#Ref<0.1217530001.50724938.164859>,...},
...}, ** Reason for termination == , ** {bad_return_value,last_stored_block_index_has_no_intersection_with_the_new_one}
2021-04-13T14:00:35.233826+00:00 [error] proc_lib.erl:508 crasher: initial call: ar_data_sync:init/1, pid: <0.780.0>, registered_name: ar_data_sync, exit: {{bad_return_value,last_s
tored_block_index_has_no_intersection_with_the_new_one},[{gen_server,handle_common_reply,8,[{file,"gen_server.erl"},{line,755}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},
{line,249}]}]}, ancestors: [ar_sup,<0.745.0>], message_queue_len: 0, messages: [], links: [<0.755.0>], dictionary: [], trap_exit: true, status: runn...
[Stage 1/3] Starting to hash
2021-04-13T14:00:35.719227+00:00 [error] supervisor.erl:701 supervisor: {local,ar_sup}, errorContext: child_terminated, reason: {bad_return_value,last_stored_block_index_has_no_int
ersection_with_the_new_one}, offender: [{pid,<0.780.0>},{id,ar_data_sync},{mfargs,{ar_data_sync,start_link,[]}},{restart_type,permanent},{shutdown,5000},{child_type,worker}] And curl localhost:2984/ gives
{"network":"arweave.testnet","version":5,"release":46,"height":0,"current":"wa1wb5Zzn-22M_SiPxjNRt4wkbCeTXi4RK8_R5UBbzFrWXbaanQQAwhohmSpavWA","blocks":0,"peers":0,"queue_length":0 ,"node_state_latency":11}
Why height is 0?
ldmberman
commented
3 years ago height is 0 because you started a new weave - it only the genesis block so far. The error occurs because there is already a weave started in this directory (_build/debug/rel/arweave), you can specify a different one via data_dir. Once you initialize a weave, the subsequent launches of the node should be with start_from_block_index instead of init.
vird
commented
3 years ago That's better
I made
rm -rf _build/debug/rel/arweave
./rebar3 as debug tar
./_build/debug/rel/arweave/bin/start init mine mining_addr w6PJGgRIk_KE0t6xEKeam3vDamx_f0hBwfCk8wPmR-Q max_miners 1 port 2984
but after 1 hour I have still height 0
Initialising RandomX dataset for fast hashing. Swap height: 0, Key: <<"QXJ3ZWF2ZSBHZW5lc2lzIFJhbmRvbVggS2V5">>. The process may take several minutes.
RandomX dataset initialisation for swap height 0 complete.
[Stage 1/3] Starting to hash
No foreign blocks received from the network or found by trusted peers. Please check your internet connection and the logs for errors.
No foreign blocks received from the network or found by trusted peers. Please check your internet connection and the logs for errors.
No foreign blocks received from the network or found by trusted peers. Please check your internet connection and the logs for errors.
No foreign blocks received from the network or found by trusted peers. Please check your internet connection and the logs for errors.@vird, hm, I think, the initial difficulty is too large, you should try the feature/testnet branch. Also, could you, please, describe your motivation behind running a test node?
vird
commented
3 years ago I'm experimenting with fair pool approach for arweave.
Version 1. Nodes (with my patch) can use option pool_address for getting a block candidate and send solutions to pool, but also they will support network (hpool does not support arweave network).
Version 2. V1 + Nodes can be paired with high performance xmrig randomx implementation (I know that arweave randomx != monero randomx). so hashing on xmrig, chunk access on arweave side, sync new chunks on arweave side
So I need fully working testnet node for that Also I preffer master branch, because I want apply patches to mainnet and test as close as possible
vird
commented
3 years ago Some investigation about inner miner calls
stage_one_thread and stage_two_thread spawn calls are present
StageOneHasher - calls fine pick_recall_byte - no calls StageTwoHasher - no calls SmallWeaveHasher - no calls
tested with default parameters and with stage_one_hashing_threads 1 stage_two_hashing_threads 1 io_threads 4 randomx_bulk_hashing_iterations 1
Something is definitely wrong with hasher if not hardfork, but init with spora. changing DEFAULT_DIFF to 6 had no effect (obviously).
ldmberman
commented
3 years ago I'm experimenting with fair pool approach for arweave.
Got it, thank you for clarifying.
What do you mean by hasher if not hardfork, but init with spora? When you compile as debug, all the fork heights are reset, so the code corresponds to the latest fork.
vird
commented
3 years ago I mean that after that reset something goes wrong and hasher don't work properly (possibly because it doesn't see any chunks)
upd. it could be problem with init+mine launch
ldmberman
commented
3 years ago @vird ah, I figured it out, add enable search_in_rocksdb_when_mining to the startup arguments. The genesis data is stored in RocksDB, whereas since 2.4.1 mining exclusively relies on the new fast storage unless explicitly told to use rocks.
vird
commented
3 years ago Yes. Now it works. For future references here is my recipe
ulimit -n 1000000
rm -rf _build/debug/rel/arweave
./rebar3 as debug tar
./_build/debug/rel/arweave/bin/start init mine mining_addr w6PJGgRIk_KE0t6xEKeam3vDamx_f0hBwfCk8wPmR-Q port 2984 \
enable search_in_rocksdb_when_mining \
stage_one_hashing_threads 1 stage_two_hashing_threads 1 io_threads 4 randomx_bulk_hashing_iterations 1my log
Initialising RandomX dataset for fast hashing. Swap height: 0, Key: <<"QXJ3ZWF2ZSBHZW5lc2lzIFJhbmRvbVggS2V5">>. The process may take several minutes.
RandomX dataset initialisation for swap height 0 complete.
[Stage 1/3] Starting to hash
Miner spora rate: 26 h/s, recall bytes computed/s: 27, MiB/s read: 0, the round lasted 15 seconds.
[Stage 2/3] Produced candidate block zlseiWNrnPjM7Clal-_jfYOK_5s9tZ20stqVNqrDay2AL1Yd2s11Fms1LTeK0Yo7 and dispatched to network.
[Stage 1/3] Starting to hash
Miner spora rate: 5 h/s, recall bytes computed/s: 5, MiB/s read: 0, the round lasted 11 seconds.
[Stage 2/3] Produced candidate block p09_1hdH_9EfoHX3SVTEFLVjITNZFphlIrHRh3m7EGaLug6_CqEenWGFJGBqBs-S and dispatched to network.
[Stage 1/3] Starting to hash Verify height curl localhost:2984
{"network":"arweave.testnet","version":5,"release":46,"height":8,"current":"i5S-k62CBDTOI2jHZn-tLq8ja9zPzWbAqBht5MfMAaHob7TpQwB8Oszl9x9SMDg3","blocks":8,"peers":0,"queue_length":0,"node_state_latency":9}
try connect + mine
also I can't found no replacement for arweave.net for testnet
try to start own weave ./bin/start-dev init port 2984 2>&1 | tee dump