Stuck 'Checking if public key is generated'

[wni-matthewcurran]

Bug Description

I installed RustDesk Server (OSS Version on Rocky Linux 9), all ports were opened using firewalld I got that going without much issue. I tested connecting to clients with the correct network configuration, I didn't have an issue. I then bought RustDesk Pro, so I attempted to convert from RustDesk Server to RustDesk Server Pro using the below reproduction steps and kept getting stuck on some variant of 'Checking if public key is generated...'. My understanding is that once rustdesk-hbbs.service runs it should auto-generate a public key and I managed to connect to clients on the OSS version so I'm really not sure what's causing this. Can someone please help? How do I get RustDesk Server Pro running?

I did attempt to uninstall RustDesk entirely to start from scratch, I'm unsure I did that right. A log of my terminal history are attached, some parts are cut out for my own privacy but I have kept it in chronological order.

I'm happy to provide further info if needed.

rustdesk-terminal-history.txt

How to Reproduce

1. Install the OSS version via the install script.
2. Attempt to convert to RustDesk Server Pro via the documented upgrade script bash <(wget -qO- https://raw.githubusercontent.com/rustdesk/rustdesk-server-pro/main/update.sh)
3. Get stuck on the below for 2+ hours.

   ++ find /var/lib/rustdesk-server -name '*.pub'
   + PUBKEYNAME=
   + '[' -z '' ']'
   + print_text_in_color '\e[0;96m' 'Checking if public key is generated...'
   + printf '%b%s%b\n' '\e[0;96m' 'Checking if public key is generated...' '\e[0m'
   Checking if public key is generated...
   + sleep 5
   + :

4. Uninstall RustDesk using the below commands.

   sudo systemctl stop rustdeskrelay.service rustdesk-hbbr.service
   sudo systemctl disable rustdeskrelay.service rustdesk-hbbr.service
   sudo systemctl stop rustdeskrelay.service rustdesk-hbbs.service
   sudo systemctl disable rustdeskrelay.service rustdesk-hbbs.service
   sudo rm -rf /var/log/rustdesk/
   sudo rm -rf /opt/rustdesk/
   sudo rm -rf /etc/systemd/system/rustdesksignal.service
   sudo rm -rf /etc/systemd/system/rustdeskrelay.service
   sudo rm -rf /etc/systemd/system/rustdesk-hbbs.service
   sudo rm -rf /etc/systemd/system/rustdesk-hbbr.service

5. Attempt to reinstall RustDesk Server Pro without conversion, just from scratch using the documented script: bash <(wget -qO- https://raw.githubusercontent.com/rustdesk/rustdesk-server-pro/main/install.sh)
6. Get stuck on the below for 2+ hours.

   Checking if public key is generated...

Expected Behavior

RustDesk Server OSS converts to RustDesk Server Pro, it doesn't get stuck on Checking if public key is generated...

Operating system(s) on local side and remote side

Rocky Linux 9

RustDesk Version(s) on local side and remote side

1.2.3

Screenshots

Additional Context

No response

[rustdesk]

763

[rso42]

The installer script has a two flaws where the installtion might get stuck at the 'Checking if public key is generated...' point.

1. The install script asks for an unpreviledged user and simply asumes that also a gropu with the same name would exist. -> suggestion for improvement: install script should check also for existance of group

2. Without an existing group with the same name as the user the 'chown' for the $RUSTDESK_INSTALL_DIR will fail. -> suggestion for improvement: install script could check for correct access rights on $RUSTDESK_INSTALL_DIR

3. Without access rights to the $RUSTDESK_INSTALL_DIR the rustdesk-hhbs.service will fail to start ('Error: error returned from database: (code: 14) unable to open database file') -> suggestion for improvement: install script should not only check for hbbr service being active but also for hbbs service before proceeding

---

manual fix:

• create a new user and group, both with the same name, 'rustdeskusr' in this example: sudo useradd -r -U -c "Rustdesk service user" -d /nonexistent -s /bin/false rustdeskusr
• restart the install script and use the newly created user