restart of enabled tor master node with new pivx-5.3.0rc1 codebase throws it to uninitialized state till started again

[eval9]

Describe the issue

valid and enabled masternode with tor address when restarted onto the new codebase pivx-5.3.0rc1 goes into non initialized set and never switches to enabled , only doing fresh new start makes that master node enabled again but that of course throws it the end of the queue for 3 Days +

Can you reliably reproduce the issue?

If so, please list the steps to reproduce below:

1. restart pivxd demon 2.logs will eventually show ManageStatus - not capable: Masternode address:port connection availability test failed, could not open a connection to the public masternode address (xxxxxxxxxxxxx.onion:51472)
2. start master node form the controller again makes master node enabled but goes for 3 days "jail" again

Expected behavior

Tell us what should happen

just changing version should keep masternode enabled without doing hard restart

Actual behavior

changing version (pivxd restart) makes masternode uninitialized requiring hard restart

What version of PIVX Core are you using?

pivx-5.3.0rc1

[furszy]

Hi this is described in the release-notes, please check #2526, inside the "P2P and network changes" section:

• The Tor onion service that is automatically created by setting the -listenonion configuration parameter will now be created as a Tor v3 service instead of Tor v2. The private key that was used for Tor v2 (if any) will be left untouched in the onion_private_key file in the data directory (see -datadir) and can be removed if not needed. PIVX Core will no longer attempt to read it. The private key for the Tor v3 service will be saved in a file named onion_v3_private_key. To use the deprecated Tor v2 service (not recommended), then onion_private_key can be copied over onion_v3_private_key, e.g. cp -f onion_private_key onion_v3_private_key.

Essentially you have to copy your v2 onion address into the new v3 file to continue running the Tor v2 addr service. Be aware that the Tor's v2 addresses EOL is close to happen, the recommendation is to move to a v3 addr once the v5.3 network upgrade gets enforced.

[furszy]

I'm going to move this section to the top of the release-notes, so it gets more attention there, with further explanations for MNs (like bolding the v2 onion addr EOL and maybe add a quick Tor v3 onion addr migration guide after the v5.3 enforcement).

Thanks for testing the rc and writing your feedback 👍 .

[eval9]

thanks, will try for now copying tor2 to tor3 however I am aware that eventually tor itself might deprecate tor2 addresses do you know safe ETA on this?, new address (tor3) generates "jail time" as its like new master node..

[eval9]

ok, copied onion_private_key to onion_v3_private_key

and I see ERROR: Read : Deserialize or I/O error - non-canonical ReadCompactSize(): iostream error Error reading mncache.dat - cached data discarded

and also

ManageStatus - not capable: Masternode address:port connection availability test failed, could not open a connection to the public masternode address (xxxxxxxxxxxxx.onion:51472)

and status shows

"Active Masternode not initialized."

so masternode failed to enable on start and needs to be started from the controller (waiting does not help eventually 2 hr rule kicks in and mn is removed)

[furszy]

will try for now copying tor2 to tor3 however I am aware that eventually tor itself might deprecate tor2 addresses do you know safe ETA on this?, new address (tor3) generates "jail time" as its like new master node..

Sure, some dates so you are fully aware of the context: 1) The Tor's final deprecation for v2 onion address is scheduled for Oct 15th. At that date, every peer running a tor v2 service will be expelled from the network. 2) PIVX v5.3 network upgrade enforcement is scheduled for Sept 9/10th (at block 3,014,000).

So, after the v5.3 enforcement, you will a month to move to a v3 onion service.

[eval9]

thanks makes sense to move to tor3 addresses but the restart and mn going to inactive state that I feel is an issue still (even if tor v3 address is used I worry that on power failure this problem will re-surface as one desires to keep em nodes enabled ...)

[eval9]

interestingly I just changed to2 to tor3 address in masternode.conf on the controller side and I see mn private key has not changed (when I was using tor2 address) so thats actually good, cause if restart would work there would be 0 interruption (and no 3 Day jail time)

yep just tried fresh new tor3 address and it never enabled (on restart)

never mind cause enforcement so bad test.. so just v2 for now.. but when its enforced restart will be a problem regardless

[furszy]

and I see ERROR: Read : Deserialize or I/O error - non-canonical ReadCompactSize(): iostream error Error reading mncache.dat - cached data discarded

This is fine if it happened at the node's first startup after the v5.3 update, the MN cache will be re-constructed from the network and stored on disk using the new addresses serialization format (v2 network addresses).

ManageStatus - not capable: Masternode address:port connection availability test failed, could not open a connection to the public masternode address (xxxxxxxxxxxxx.onion:51472)

This logging is mostly useless for now, don't give it any attention. It's not a MN blocking reason.

"Active Masternode not initialized." so masternode failed to enable on start and needs to be started from the controller (waiting does not help eventually 2 hr rule kicks in and mn is removed)

hmm, I'm most likely missing some information here but probably your node needed more time to sync the MN list. The current sync process "slowly" syncs up the missing MNs after the initial synchronization (this is something that will be completely changed with the deterministic masternodes implementation in v6.0).

just tried fresh new tor3 address and it never enabled (on restart) never mind cause enforcement so bad test.. so just v2 for now.. but when its enforced restart will be a problem regardless

Yes, that is fine. MNs running a v3 onion addrs are not accepted before the v5.3 enforcement. The addr serialization is completely different, non-upgraded nodes will not be able to parse the masternode broadcast message.

About the restart need after the v5.3 for MNs running v2 onion addrs: yeah, you will need to broadcast the start message signaling the new v3 onion addr service after the enforcement. Not much that can be done there for now in the current system. One of the many reasons why v6.0 will be really nice is that, with the deterministic Masternodes work, you will be able to update the service address only (and much more), without requiring to broadcast the "start message" again.

[eval9]

thanks thats awesome (deterministic setup) I guess I will need to take the jail time for now, have em all start by hand from controller and then wait for 6.0 , its all good.. and expected ,, great progress

[furszy]

cool, if you have any other doubt or manage to break something there keep shooting, here to help and correct any problem before the final release.

[furszy]

Follow up to this convo, added the following paragraph to the "How To Upgrade" section in the v5.3 release notes (#2526):

Important note for Masternodes running over Tor (v2 onion address):

Before starting the node, copy the content of the `onion_private_key` file, located inside the data directory into a new `onion_v3_private_key` file inside the same directory.
On linux: `cp -f onion_private_key onion_v3_private_key`.
If the `onion_v3_private_key` file already exist, replace the content with the content of the `onion_private_key` file.

This will allow you to bypass the v2 onion address deprecation and continue running the MN over Tor for now.
Be aware that the Tor network is completely removing v2 onion addresses support starting from Oct 15th.
After the v5.3 network upgrade enforcement, the MN will need to be migrated to run on a Tor v3 onion address (Update window Sept 15th - Oct 15th).
If it's not done on time, the node will drop off the network.

[eval9]

thanks thats awesome (deterministic setup) I guess I will need to take the jail time for now, have em all start by hand from controller and then wait for 6.0 , its all good.. and expected ,, great progress

I feel we can mark this ticket as designed, despite the fact that master nodes on restart can and are going to inactive state (its sort of unpredictable when they get enabled sometimes immediately after restart sometimes maybe even over 2 hours after which they get removed..)

so I feel 6.0 will really address this as I understand your comment so one just have to make sure servers have continuous supply of power