search

gitbls / sdm

Raspberry Pi SD Card Image Manager
MIT License
471 stars 48 forks source link

Example of installing SyncThing through SDM? #247

Closed rvaneerd closed 3 months ago

rvaneerd commented 3 months ago

Hi there,

As I'm trying to re-image my RPIs, I was wondering whether someone has an example on how to install SyncThing through sdm.

I know I can install "syncthing" (and it's prerequisite apt-transport-https) through the apps functionality but after installing apt-transport-https, it seems I somehow need to add a GPG Key to the keyrings directory.

I'd really appreciate any help (or an alternative method) to get syncthing installed.

Regards, Rob

gitbls commented 3 months ago

Helpful to work with a known good install guide or script.

Can you point me to a syncthing install guide that shows all the steps that you're referring to?

rvaneerd commented 3 months ago

Hi @gitbls , Thanks for your swift response. I was referring to this one: https://pimylifeup.com/raspberry-pi-syncthing/

And this one (looking more like an official one) also refers to the GPG key: https://apt.syncthing.net

Regards, Rob.

gitbls commented 3 months ago

Looks pretty straightforward to build a syncthing plugin that will at least be able to install syncthing. Clearly any GUI configuration will have to be done outside of sdm. Will need to sort how much configuration before that can be automated.

Stay tuned.

rvaneerd commented 3 months ago

Looks pretty straightforward to build a syncthing plugin that will at least be able to install syncthing. Clearly any GUI configuration will have to be done outside of sdm. Will need to sort how much configuration before that can be automated.

Stay tuned.

Thanks @gitbls! I do understand the Configuration part (either in GUI or through a terminal session). I'll be patient ;-) Regards, Rob.

gitbls commented 3 months ago

I've posted the first version of the syncthing plugin. If you download and run EZsdmInstaller you'll pick it up.

Also, FYI I've added this issue on syncthing to see if the configuration during customization can be further improved.

Please let me know how it works for you.

rvaneerd commented 3 months ago

I've posted the first version of the syncthing plugin. If you download and run EZsdmInstaller you'll pick it up.

Also, FYI I've added this issue on syncthing to see if the configuration during customization can be further improved.

Please let me know how it works for you.

@gitbls, Thanks a lot! As I'm currently on holiday (without one of my Pi's 🫣🥴) I can't check it out right now. I will prepare though and test by the end of this week.

Regards, Rob

rvaneerd commented 3 months ago

@gitbls

I've got my RPi (Zero) up and running including syncthing! The only thing missing was that I had to look for a way to have it autostarted as it apparently did not. This may be due to the fact that upon initial start, syncthing is doing some initial configuration like database updates etc.

Once the initial run was completed, I could enable it using: sudo systemctl enable syncthing@rob.service (as I used rob is my user).

Once it was up and running I was also able to change it's name (from the one configured in sdm's syncthing plugin) into the hostname (for my personal clarity).

If you have any thoughts about enabling the service then I'm happy to play around again.

If I'm right (after further examining the syncthing plugin), I forgot to specify "enablesvc=y" If I would have done that, the service would have been enabled from start. Is that assumption correct?

If so, this issue can be closed and I will reopen a new one when I run into new thoughts.

Regards and thanks for the swift and good support! Rob.

gitbls commented 3 months ago

Re autostart: If you use the argument enabelsvc with the syncthing plugin that will enable the service as part of the first system boot, so it should come up started on the reboot at the end of the FirstBoot process.

However, if there is still configuration to be done, that could certainly cause the service to fail.

The hostname should be changeable in the syncthing plugin via the argument synchost if you know it, of course.

One way to handle situations like this is to NOT use the plugin during a --customize, but instead to invoke it during --burn, when the actual target hostname is known. You'll still need to use the synchost argument, but if you set it correctly you shouldn't need to change it again. This has the added benefit of using a common customized IMG across multiple Pi systems, and do specific per-host customization at the last minute (when burned).

A question for you: Was there any syncthing configuration that you had to do using the GUI after the system was up and running (other than the hostname)? If there was, could you list those things so I can see about adding support in the plugin for them? Thx!

rvaneerd commented 3 months ago

With regards to the enablesvc argument: I had it configured just as syncthing:"**_enablesvc_**|gui-address=0.0.0.0:8384|...... Was that sufficient? Or should it have been **_enablesvc=y_**?

As for the hostname, that is the same one that would be used when burning the SD-card. So, if there is anyway to get that implemented in the installation?

I'm trying to bend my head around what you wrote in the bigger part of your response. Is that something that I could have implemented in my configuration? Or is that something for a future improvement in the plugin code?

As to your question: I think, the only thing I added (besides making it start automatically) for configuration was the synchost name using the GUI. But I will rebuild my SD-Card and start over while making notes. That is the good thing about playing around with a spare Raspberry Pi ;-)

gitbls commented 3 months ago

With regards to the enablesvc argument: I had it configured just as syncthing:"**_enablesvc_**|gui-address=0.0.0.0:8384|...... Was that sufficient? Or should it have been **_enablesvc=y_**?

As for the hostname, that is the same one that would be used when burning the SD-card. So, if there is anyway to get that implemented in the installation?

I'm trying to bend my head around what you wrote in the bigger part of your response. Is that something that I could have implemented in my configuration? Or is that something for a future improvement in the plugin code?

As to your question: I think, the only thing I added (besides making it start automatically) for configuration was the synchost name using the GUI. But I will rebuild my SD-Card and start over while making notes. That is the good thing about playing around with a spare Raspberry Pi ;-)

enablesvc by itself is sufficient. I don't see where you used the synchost argument in your syncthing plugin invocation. I'll look into whether the syncthing plugin can get the hostname used when burning when I free up next week.

Not sure which part you're trying to get your head around. Can you expand on this?

gitbls commented 3 months ago

OK, just did a test and indeed, there's something awry with the plugin enabling the service. Investigating.

I wasn't able to repro your issue with the hostname. I set it on the --customize command and then burned it, using the same --hostname hostname as I used for the syncthing plugin, and syncthing came up with the correct hostname. Looking at the code it's not clear to me how you'd run into a problem with the hostname if you always specify it.

And yes, looking into getting the hostname from sdm as well, so it only needs to be set in one place

gitbls commented 3 months ago

Apologies. The service autostart wasn't working correctly. Still looking at improving the hostname setting, then will release an updated version.

rvaneerd commented 3 months ago

Ok, I reburned the image generated earlier (so no re-generate) and restarted my RPi with it. Thinks I checked: sudo systemctl status syncthing@rob resulted in:

○ syncthing@rob.service - Syncthing - Open Source Continuous File Synchronization for rob
     Loaded: loaded (/lib/systemd/system/syncthing@.service; disabled; preset: enabled)
     Active: inactive (dead)
       Docs: man:syncthing(1)

Then I started the service: sudo systemctl start syncthing@rob

and rechecked it's status: sudo systemctl status syncthing@rob Which now resulted in:

● syncthing@rob.service - Syncthing - Open Source Continuous File Synchronization for rob
     Loaded: loaded (/lib/systemd/system/syncthing@.service; disabled; preset: enabled)
     Active: active (running) since Sat 2024-08-17 10:32:29 CEST; 43s ago
       Docs: man:syncthing(1)
   Main PID: 844 (syncthing)
      Tasks: 15 (limit: 387)
        CPU: 7.055s
     CGroup: /system.slice/system-syncthing.slice/syncthing@rob.service
             ├─844 /usr/bin/syncthing serve --no-browser --no-restart --logflags=0
             └─850 /usr/bin/syncthing serve --no-browser --no-restart --logflags=0

Aug 17 10:32:34 RvE-RPiZero syncthing[844]: [JCKEA] INFO: Using discovery mechanism: IPv6 local multicast discovery on address [ff12::8384]:21027
Aug 17 10:32:34 RvE-RPiZero syncthing[844]: [JCKEA] INFO: Creating new HTTPS certificate
Aug 17 10:32:34 RvE-RPiZero syncthing[844]: [JCKEA] INFO: GUI and API listening on [::]:8384
Aug 17 10:32:34 RvE-RPiZero syncthing[844]: [JCKEA] INFO: Access the GUI via the following URL: http://127.0.0.1:8384/
Aug 17 10:32:34 RvE-RPiZero syncthing[844]: [JCKEA] INFO: My name is "mysyncserver"
Aug 17 10:32:53 RvE-RPiZero syncthing[844]: [JCKEA] INFO: quic://0.0.0.0:22000 detected NAT type: Port restricted NAT
Aug 17 10:32:53 RvE-RPiZero syncthing[844]: [JCKEA] INFO: quic://0.0.0.0:22000 resolved external address quic://95.99.17.100:22000 (via stun.syncthing.net:3478)
Aug 17 10:32:54 RvE-RPiZero syncthing[844]: [JCKEA] INFO: New external port opened: external TCP address(es) [95.99.17.100:28269] to local address [::]:22000.
Aug 17 10:32:54 RvE-RPiZero syncthing[844]: [JCKEA] INFO: New external port opened: external TCP address(es) [95.99.17.100:19108] to local address [::]:22000.
Aug 17 10:32:54 RvE-RPiZero syncthing[844]: [JCKEA] INFO: Detected 2 NAT services

After a reboot of the RPi, the service was (again / still) not started. So I (re-)enabled it: sudo systemctl enable syncthing@rob resulting in: Created symlink /etc/systemd/system/multi-user.target.wants/syncthing@rob.service → /lib/systemd/system/syncthing@.service.

From the GUI I have performed the following actions / tasks:

Conclusion

So, apart from service configuration (and the required GUI actions), syncthing seems to be installed properly!

Additional feature?

One additional thing from the GUI, which might be added to the sdm syncthing configuration is the accept / deny of sending user statistics. Initially, the config.xml contains the following line in the <options> section: <urAccepted>0</urAccepted>

After denying the request through the GUI, the line was changed to: <urAccepted>-1</urAccepted>

Then I reburned the SD-Card again and started my RPi. Before starting syncthing though, I changed the mentioned line in the config.xml to reflect the "deny" status: <urAccepted>-1</urAccepted> Then after starting the syncthing service, I did not get prompted to accept / deny the sending of user statistics.

Can this be added to the syncthing plugin? I think valid values for this parameter are:

Or do you prefer a separate issue for this?

Regards, Rob.

gitbls commented 3 months ago

As I mentioned I'm re-looking at the automatic startup, so will get to this once I've finished that work. In the meantime, don't need any more input on service enabling/starting (or not). Current code is just...wrong.

And yes, the capability to accept/deny user stat sending is important. Will add.

gitbls commented 3 months ago

I just pushed a new syncthing plugin. If you update your sdm with EZsdmInstaller you'll get the new one. There are some significant changes:

All documented at https://github.com/gitbls/sdm/blob/master/Docs/Plugins.md#syncthing

Please LMK how this works for you.

rvaneerd commented 3 months ago

Thanks a lot for all your effort. Will experiment further and keep you posted.

gitbls commented 3 months ago

FYI, just posted an updated syncthing plugin that improves (even further!) the enablement of the syncthing user service. Took me a while to sort out the systemctl magic for that.

rvaneerd commented 3 months ago

Great! Will experiment soon with it. I might investigate what you've done as I may need to reuse part of it for my zerotier plugin #252.

gitbls commented 3 months ago

My basic approach to building an automated installer (plugin) is to try simple things first, work on all the issues that show up in either the sdm log or the system journal (journalctl). Need to check both!

syncthing was a bit special because the per-user config file needed some special handling. Also, getting the service startup working correctly was a bit of hassle, but I think it's good now. LMK if you have any questions on what the syncthinng plugin is doing

Happy to help you sort out issues with zerotier so you don't over- or under- rotate on the code.

rvaneerd commented 3 months ago

My latest build (mentioned in #252) also included your latest changes to SyncThing. This was also successfull and sufficient for my use!

So thanks a lot!

gitbls commented 3 months ago

Perfect. Thanks for confirming. Closing this issue. Of course, open a new issue if you run into something new.