search

Ylianst / MeshAgent

MeshAgent used along with MeshCentral to remotely manage computers. Many variations of the background management agent are included as binaries in the MeshCentral project.
https://meshcentral.com
231 stars 89 forks source link

MeshAgent on read only file system #14

Open davidovv opened 5 years ago

davidovv commented 5 years ago

I installed MeshAgent on raspberry pi device. The device has read only file system (to protect sd cards from corruption due to power loss or unproper shutdown) The MeshAgent doesn't connect to MeshCentral when pi has read-only file system. How can i start MeshAgent without the need to write on file system.

GQGibbs commented 4 years ago

I also have this same problem on Rasp-Pi Buster. The mesh agent is confirmed active when the system boots in read only mode but the server (MeshServer=wss://meshcentral.com:443/agent.ashx) never receives confirmation. When I restore the write file system, the agent is immediately detected on the server.

PRETTY_NAME="Raspbian GNU/Linux 10 (buster)" NAME="Raspbian GNU/Linux" VERSION_ID="10" VERSION="10 (buster)" VERSION_CODENAME=buster ID=raspbian ID_LIKE=debian HOME_URL="http://www.raspbian.org/" SUPPORT_URL="http://www.raspbian.org/RaspbianForums" BUG_REPORT_URL="http://www.raspbian.org/RaspbianBugs"

● meshagent.service - MeshCentral Agent Loaded: loaded (/lib/systemd/system/meshagent.service; enabled; vendor preset: enabled) Active: active (running) since Mon 2020-05-11 02:50:26 EDT; 15s ago Main PID: 20772 (meshagent) Memory: 244.0K CGroup: /system.slice/meshagent.service └─20772 /usr/local/mesh/meshagent

Restore write O/S

sudo mount -o remount,rw / ; sudo mount -o remount,rw /boot

krayon007 commented 4 years ago

I'll look into this. I think it's failing to connect because the agent cant open the DB because it's asking for RW.

In your usage, are you setting up the agent in readwrite first before changing to read-only? Because the agent needs to save some basic server and certificate info into the DB when it is first setup.

GQGibbs commented 4 years ago

This helps confirm your thinking (run in RO mode): ./meshagent state MeshCentral2 Agent Unable to open database (1/10)... Unable to open database (2/10)... Unable to open database (3/10)... Unable to open database (4/10)... Unable to open database (5/10)... Unable to open database (6/10)... Unable to open database (7/10)... Unable to open database (8/10)... Unable to open database (9/10)... Unable to open database (10/10)... Unable to open database.

I'll uninstall and then reinstall in RW mode and post the results.

GQGibbs commented 4 years ago

After the uninstall/install sequence (remaining in RW mode):

Agent is online

info Current Core: MeshCore CRC-1221220589 Agent Time: 2020-05-11 08:23:02.045-04:00. User Rights: 0xffffffff. Platform: linux. Capabilities: 15. Server URL: wss://meshcentral.com:443/agent.ashx. OS: Raspbian GNU/Linux 10 (buster). Modules: amt, apfclient, monitor-border, power-monitor, sysinfo, wifi-scanner-windows, wifi-scanner. Server Connection: true, State: 1. lastMeInfo: null. X11 support: true.

Still getting the same error in RW mode even though the server detects the agent so its significance seems diminished:

./meshagent state
MeshCentral2 Agent
Unable to open database (1/10)...
Unable to open database (2/10)...
Unable to open database (3/10)...
Unable to open database (4/10)...
Unable to open database (5/10)...
Unable to open database (6/10)...
Unable to open database (7/10)...
Unable to open database (8/10)...
Unable to open database (9/10)...
Unable to open database (10/10)...
Unable to open database.
● meshagent.service - MeshCentral Agent
   Loaded: loaded (/lib/systemd/system/meshagent.service; enabled; vendor preset: enabled)
   Active: active (running) since Mon 2020-05-11 08:22:02 EDT; 7min ago
 Main PID: 1354 (meshagent)
   Memory: 5.0M
   CGroup: /system.slice/meshagent.service
           └─1354 /usr/local/mesh/meshagent

Reboot into RO mode and the server fails to detect the agent even though it's running (note the small amount of memory it reports using vs. running in RW mode)

● meshagent.service - MeshCentral Agent
   Loaded: loaded (/lib/systemd/system/meshagent.service; enabled; vendor preset: enabled)
   Active: active (running) since Mon 2020-05-11 08:38:13 EDT; 3s ago
 Main PID: 2217 (meshagent)
   Memory: 172.0K
   CGroup: /system.slice/meshagent.service
           └─2217 /usr/local/mesh/meshagent

Change to RW mode and the server instantly recognizes the agent.

davidovv commented 4 years ago

I must say that i am surprised and confused that after i posted this issue, and there was no response for almost one year, first comment on it gets response in 2 hours. Did I do something wrong way?

Anyway, i installed the agent on RW system, and later locked the system to be RO, Meshagent didn't work in RO, but changing it back to RW works

I also thought the problem is that opening the database in RW mode fails, so that a fix could be after failing to open in rw mode, agent could try to open in ro mode, if it doesn't need new information from server to store in database, but then there are some server updates that would probably be affected by this idea so i gave up

krayon007 commented 4 years ago

No, you didn't do anything wrong, I just missed the original notification, so when someone commented on it, I actually got that notification...

I did happen to implement a readonly mode for the database a few weeks ago. It's currently only used for the installer, but it would be easy to change it to retry the database in readonly mode if it can't open the db for RW. I just need to chat with @Ylianst over how to support read only filesystems, becuase while it would work to simply retry in readonly mode, we may only want to do that if the agent is configured to run that way, so we don't run into consistency issues where the agent opens up in readonly mode when it wasn't expected, causing the agent to throw away any changes that couldn't be written to the db, etc...

But once we sort out the usage, it should be pretty trivial to support...

krayon007 commented 4 years ago

I'll probably want to pass that information up to the server too, so the server can mark which agents are in readonly mode, since that will affect how auto-updates work...

frakman1 commented 2 years ago

I think it's reasonable to expect the device to be in read-write mode when installing the agent. I wouldn't expect it to install otherwise just like I wouldn't expect installing any other application to work. However, after initial installation and setup, reverting to read-only should be supported even across reboots.

Jamesits commented 1 year ago

Is it possible to split the moving parts and static ones? I'm using a similar readonly disk setup but have alternate persistent state storage available for writing database files. In my use case, the readonly disk is replicated into a lot embedded devices so it is impossible to keep any machine-specific database inside the disk image.

I currently use the following workaround:

[Unit]
Description=meshagent background service

[Service]
StateDirectory=mesh_services/meshagent
StateDirectoryMode=0700
WorkingDirectory=/var/lib/mesh_services/meshagent

ExecStartPre=-/bin/cp -an -- /usr/libexec/meshagent /var/lib/mesh_services/meshagent/
ExecStartPre=-/bin/cp -an -- /usr/share/meshagent/. /var/lib/mesh_services/meshagent/
ExecStart=/var/lib/mesh_services/meshagent/meshagent --installedByUser=0 --no-embedded=1 --copy-msh=0

Restart=always
RestartSec=3

[Install]
WantedBy=multi-user.target

The only drawback of this method is that it creates a new computer on MeshCentral on every reboot.