search

zerotier / ZeroTierOne

A Smart Ethernet Switch for Earth
https://zerotier.com
Other
14.16k stars 1.65k forks source link

Filesystem layout of ZT_HOME is a pain for Read Only uses. #2209

Open laduke opened 7 months ago

laduke commented 7 months ago

zerotier-one constantly makes writes to some of these files. The files are mixed in with the more static files.

Embedded users have issues with SD cards getting burnt out, or the fs get re-mounted RO some time after boot and then zerotier-one chokes in some strange way.

Current layout:

tree /var/lib/zerotier-one/
/var/lib/zerotier-one/
├── authtoken.secret
├── controller.d
│   ├── network
│   └── trace
├── identity.public
├── identity.secret
├── metrics.prom
├── metricstoken.secret
├── moons.d
├── networks.d
│   ├── <network1>.conf
│   ├── <network1>.local.conf
│   ├── <network2>.conf
│   └── <network2>.local.conf
├── peers.d
│   ├── 1111111111.peer
│   └── 2222222222.peer
├── planet
├── zerotier-cli -> /usr/sbin/zerotier-one
├── zerotier-idtool -> /usr/sbin/zerotier-one
├── zerotier-one -> /usr/sbin/zerotier-one
├── zerotier-one.pid
├── zerotier-one.port

Things that get written to : networks.d/.conf peers.d/* metrics.prom planet (if rarely)

pid and port on start up

authtoken and identity on First start.

if you join a network or change network options, networks.d/.local.conf local.conf

I think the three categories are something like: config cache (ephemeral stuff) identity and token (this is just autogenerated config?)

If we could separate some these types of files someday, that'd be nice

this might be helpful to some users: https://github.com/azlux/log2ram

joseph-henry commented 7 months ago

The log2ram idea is neat but I could build in a local.conf flag like embedded-write-mode: true|false where it will only write network/peer config data during initial startup and shutdown of the daemon. Does that feel like a passable solution?

laduke commented 7 months ago

That'd be cool. It would help with running on a rw SD card. I suspect we need more research or input to solve the whole thing. I wrote this ticket as a more of a future note and place for discussion.