RangeError: Maximum call stack size exceeded

[gdiscord]

Hitting this error at specific block height when trying to setup the browser for PIVX blockchain. The block height is 631919.

Looking at the crytoid explorer it shows this particular block has a huge number of utxos - see

https://chainz.cryptoid.info/pivx/block.dws?e901a42fe4e3840cf4a90711c4ae0000df3f266087bb2cc9fbcd6ee68d1ffb1c.htm

Could you please give some pointers on how to fix the error?

Below is the error.

function processVoutAddresses(address_list, vout_value, arr_vout, cb) { ^

RangeError: Maximum call stack size exceeded at processVoutAddresses (/PIVX/explorer/lib/explorer.js:80:30) at /PIVX/explorer/lib/explorer.js:1143:13 at Object.next (/PIVX/explorer/lib/explorer.js:806:13) at /PIVX/explorer/lib/explorer.js:1147:20 at /PIVX/explorer/lib/explorer.js:103:18 at Object.convert_to_satoshi (/PIVX/explorer/lib/explorer.js:141:12) at /PIVX/explorer/lib/explorer.js:100:24 at /PIVX/explorer/lib/explorer.js:1059:14 at Object.next (/PIVX/explorer/lib/explorer.js:786:13) at /PIVX/explorer/lib/explorer.js:1055:14 at Object.next (/PIVX/explorer/lib/explorer.js:806:13) at Object.syncLoop (/PIVX/explorer/lib/explorer.js:831:10) at Object.is_unique (/PIVX/explorer/lib/explorer.js:1048:20) at processVoutAddresses (/PIVX/explorer/lib/explorer.js:90:20) at /PIVX/explorer/lib/explorer.js:1143:13 at Object.next (/PIVX/explorer/lib/explorer.js:806:13)

Node.js v20.10.0

[joeuhren]

First of all, the explorer doesn't yet support zerocoin or SHIELD transactions (I would love to add support for these in the near future, but it's very low priority for me at the moment without some financial incentive). If you are trying to sync the actual Pivx blockchain as opposed to a clone or fork which never used private transactions then you will undoubtedly have problems with those private transactions not adding up correctly or other weird behavior unless you plan to add support for those protocols yourself.

That being said, you are correct about the issue being related to "too many" utxos which essentially cause the sync script to run out of memory. This is a known issue which is documented right at the top of the "Known Issues" section of the README:

Basically, add --stack-size=[SIZE] to your crontab for the block sync and change [SIZE] to a larger number than the default, which will cause the script to utilize more memory each time it is run. For example:

*/1 * * * * cd /path/to/explorer && /path/to/node --stack-size=10000 scripts/sync.js update > /dev/null 2>&1

[gdiscord]

Thank you for the quick response. Didn't realise the "Known Issues" documentation prior to posting my issue. But shortly after posting I went ahead and researched the issue through the search engines and read about the v8 stack size options.

Its seems to be that node removed this option, and therefore setting it doesn't have any effect whatsoever.

In fact, in the prestart.js script that starts the explorer and I see --stack-size=10000 is already there.

// Setting the NODE_ENV variable is more easily done from here seeing at the syntax changes slightly depending on operating system execSync(${(process.platform == 'win32' ? 'set' : 'export')} NODE_ENV=${node_env} && pm2 ${startOrReload} ./bin/instance -i 0 -n explorer -p "./tmp/pm2.pid" --node-args="--stack-size=10000" --update-env, {stdio : 'inherit'});

which seems to add some weight to me guess.

But I will still go ahead and try different settings.

In the meantime I found some interesting conversation here on the topic (albeit a few years old already), wherein it is suggested to wrap the cb functions in setTimeout or equivalent instead. Perhaps worth a look into?

https://github.com/nodejs/node/issues/6360

[joeuhren]

In fact, in the prestart.js script that starts the explorer and I see --stack-size=10000 is already there.

The explorer is broken into several different components, or more simply, the webserver and the sync cmds. The webserver is the part that already has the stack size set to 10000 by default in the prestart.js or in the package.json, depending on which method you use to run the explorer webserver.

As I indicated above, you need to apply the --stack-size=[SIZE] to the block sync cmd directly since it has its own stack size which defaults to whatever your machine decides is the default and can differ from computer to computer. If --stack-size=10000 is still too small, try a higher number until it works. It might be better/easier to run the sync cmd manually at first until you find a value that seems to work with your blockchain and then update your crontab.

[gdiscord]

Indeed, I can confirm the solution you put forward overcomes the error. node --stack-size=10000 scripts/sync.js index update

It is carrying on with the sync now; will report back with later findings.

Thanks

[gdiscord]

The sync carried on fine until it reached block height 248331 and then this error was displayed in the log output:

No document found for deletion. Segmentation fault

I then tried the sync again with stack_size values higher than 10000, all the way up to 1000000, but each produced the same result:

Syncing blocks.. Please wait.. No document found for deletion. Segmentation fault

The block shows fine on the cryptoid explorer, albeit with a around 1000 utxos.

https://chainz.cryptoid.info/pivx/block.dws?37ef1a43032683e192a629358ec4d0098399533c9107fbfe5a627712901dcec5.htm

Is it the case that the eiquidus explorer is not able to process this many utxos, and therefore cannot be used for pivx?

thanks

[joeuhren]

I don't believe I've ever seen or encountered that error before, nor has anyone else ever reported it until now. The fact that it threw a segmentation fault seems to suggest it encountered some unexpected data which it wasn't able to handle properly. My best guess is its due to 1 of the 2 unsupported protocols I mentioned above, but I'm just guessing.

Can I ask why you are so bent on trying to sync a pivx explorer with eiquidus? Even if you get passed the error, the block, tx and address data cannot be trusted due to the existence of private transactions that will either be skipped or not synced in full so I'm a little baffled by why you are trying to do so.

For now, the main use case is bitcoin-clones that do not have non-standard txs. It is even compatible with zksnarks, so coins like zcash should be syncable, although to be honest I haven't kept up with any new developments in the last couple years, so support for newer versions of the zksnarks protocol may require some tweaking.

Point is, you are trying to sync with one of the few coins that won't work too well so it doesn't surprise me there are problems.

[gdiscord]

We need to sync pivx because we are trying to test some application integration with pivx, and need to utxos. I'm aware about the zerocoin/znarks, but we are not concerned about that for this particular test.

In the past I've actually synced a few pivx clones successfully, even those that had zero coin transactions in the chain.

[joeuhren]

The reason the explorer doesn't crash on zerocoin txs is because very early on I was planning to support it but decided against it due to other pressing matters and then it got deprecated so I never returned to it. There is code in the explorer that specifically checks for zerocoin and completely skips it:

No effort has been made to skip SHIELD txs in this manner and I have not studied how they differ from other txs so right now its very possible they could crash the explorer if not handled correctly.

I don't have time to dig into it right now but based on pivx block height 248331, I can't really see that there are any non-standard txs in that block so it's also possible there is some other bug or issue being encountered. It is a bit strange if its a general error since there are hundreds of other coins using the explorer and it has never come up as an issue before, but it is possible there is some incorrect handling of the many utxos or some other unforeseen problem.

It will take me many hours to set up a pivx wallet and sync to that block to do some debugging, but I can put it on my list of things to do to and let you know what I find when I get around to it.

[gdiscord]

Ok, that sounds fine. In the meantime I'll be digging around to see if I come up with anything.

There are several possibilities here - including native libraries that may be buggy.

[gdiscord]

Using segfault-handler, this is what I have found so far

_PID 2012249 received SIGSEGV for address: 0x1 explorer/node_modules/segfault-handler/build/Release/segfault-handler.node(+0x372d)[0x7efc3514d72d] /lib/x86_64-linux-gnu/libc.so.6(+0x42520)[0x7efc37a5e520] explorer/node_modules/segfault-handler/build/Release/segfault-handler.node(_Z22segfault_stack_frame_1v+0x28)[0x7efc3514d7c8] explorer/node_modules/segfault-handler/build/Release/segfault-handler.node(+0x2a60)[0x7efc3514ca60] node(_ZN2v88internal25FunctionCallbackArguments4CallENS0_15CallHandlerInfoE+0x12f)[0xf29f4f] node[0xf2a7bd] node(_ZN2v88internal21Builtin_HandleApiCallEiPmPNS07IsolateE+0x115)[0xf2ac85] node[0x1934df6]

There is a reference to the stack_frame, function callback arguments and handler, and internal api call.

hopefully that can help narrow the field of view.

[gdiscord]

Please take a look at this when you have a moment:

https://github.com/team-exor/eiquidus/blob/ed8d7a59649be31de40d6fc6f4edc5889093aa8f/scripts/sync.js#L120C21-L122C64

seems to me trying to save a null tx to the database is not what you meant to do here?

Thinking it should just be: if (tx_deleted ) { ...

[gdiscord]

From further debugging this is what I have found

when I change that conditional statement to _if (txdeleted ) then it is possible to advance the sync until the next segfault.

**================ if (tx_deleted || !tx) { =========================

2519344: 679e7ee57cf0f99571aee79b3f9ca858284b5863d963f728159c43eb5b0890f6 Segmentation fault ========================**

then I revert the conditional statement to the original _if (txdeleted || !tx) { , which the results in the code section that skips adding the current tx

_// skip adding the current tx setTimeout(function() { tx = null; // check if the script is stopping if (stopSync && check_only != 2) { // stop the loop nexttx({});...

================= if (tx_deleted){ =================== gdiscord console log ====> skip adding the current tx = tx=null tx_deleted=false ==============================

then rinse and repeat.

With the above rudimentary way I am able to get the sync to progress. But obviously it is not the best.

Can you shed some light on why the above behaviour is happening? My suspicion is that there is a callback timer happening somewhere.

async is hard to debug!

thanks

[gdiscord]

Also need to add that the segfaults happen at blocks that have the same attributes - many utxos

examples

pivx block 2445309 pivx block 2446472 pivx block 2447586

if you checked those blocks on cryptoid explorer you'd see what I mean.

for all other blocks where the source/inputs has even many addresses they don't segfault.

[gdiscord]

The explorer has now synced successfully from genesis up to the block height when the sync first started on 27th December (block 4190399 with Block Hash: f801f396e83853af6f7a0e117afc3aa923034d50b05209f92ff72a53f853afb6) , but it as stopped and not fetching the blocks after that initial highest block.

I have run npm sync-blocks and node scripts/sync.js update, but none seems to work.

Can you advise on what to do to get the new blocks coming in?

thanks

[joeuhren]

I have run npm sync-blocks and node scripts/sync.js update, but none seems to work.

Those are the same cmds but one is an alias for the other. What is the output of the sync-blocks cmd if you run it manually? It should give you an error msg if there is a problem or tell you which block it is currently working on.

[gdiscord]

These are the last few lines of the output when I do npm run sync-blocks:

_setting 'api_cmds.rpc_concurrenttasks' is missing. Loading default value. Script launched with pid: 2157810 Syncing blocks.. Please wait.. 4190398: Checking for forks... Finished looking for forks Block sync complete (block: 4190399)

[gdiscord]

Actually I it turns out the pivx daemon itself is the source of the issue. I'll investigate further and update.

[joeuhren]

The block sync seems to be working based on that output. Does it continuously give you the same msg everytime you try to run it, like it's stuck on the same block? If so, is your pivx wallet stuck as well? The explorer is pretty dumb as it just gets data from the wallet and saves it to mongo database for easier retrieval. If the sync seems to be stuck, I would check your wallet and make sure its moving since the first thing it basically just gets the current block count from the wallet and then tries to sync to that number.

[joeuhren]

Actually I it turns out the pivx daemon itself is the source of the issue. I'll investigate further and update.

You beat me to it, but yeah, you just need to get the wallet moving first.

[gdiscord]

All working good now. For some reason the wallet was rejecting all the hashes, even though they were valid. Pivx support guy said this was possible sometimes when you happen to catch/connect to a node that was on a fork.

In any case, it's all working fine now.

Thanks.

[gdiscord]

Closing the case.