File Ownership Issue in 'send collectd configuration to device'

[ButKor]

Describe the bug I am experiencing an unexpected behaviour when updating a certain configuration file (/etc/collectd/collectd.conf) via thin-edge from Cumulocity. The same operation succeeds only every second try, the others in between failing with:

config-manager failed downloading a file: Error while performing a file operation: Could not move file from "/tmp/collectd.conf.tmp" to "/tmp/collectd.conf": Failed to change owner: "/tmp/collectd.conf". 

See reproduction steps to make the problem clear.

To Reproduce

1. Have a Cumulocity tenant and bootstrap a thin-edge device via https://github.com/thin-edge/tedge-demo-container
2. Go to your Device in Cumulocity and check current configuration of the tedge-configuration-plugin:

   files = [
   { path = '/etc/tedge/tedge.toml', type = 'tedge.toml', user = 'tedge', group = 'tedge', mode = 0o444 },
   { path = '/etc/tedge/plugins/tedge-log-plugin.toml', type = 'tedge-log-plugin', user = 'tedge', group = 'tedge', mode = 0o644 },
   { path = '/etc/tedge/system.toml', type = 'system.toml', user = 'tedge', group = 'tedge', mode = 0o444 },
   { path = '/etc/mosquitto/conf.d/mosquitto.conf', type = 'mosquitto.conf', user = 'root', group = 'root', mode = 0o644 },
   { path = '/etc/collectd/collectd.conf', type = 'collectd.conf', user = 'root', group = 'root', mode = 0o644 },
   { path = '/etc/c8y-command-plugin/env', type = 'shell', user = 'tedge', group = 'tedge', mode = 0o644 },
   ]

3. Within Configuration tab select collectd and get snapshot from Device. Click on save to repository once it's there.
4. Stay on Configuration tab and send the collectd file (which you've just saved to repo) to the Device:
   • First try: it fails with Failed to change owner: "/tmp/collectd.conf"
   • Second try (on same device and same file): Now the operation succeeds.
   • Third try: it fails with Failed to change owner: "/tmp/collectd.conf"
   • Fourth try: succeeds ...and so on.

Environment (please complete the following information):

• thin-edge.io version 1.3.0
• Environment: reproducible in tedge-demo-container, also reproducible on a Pi5

[Bravo555]

First try fails because:

1. collectd.conf is marked in tedge-configuration-plugin.toml as being owned by root
2. We try to chown root /tmp/collectd.conf, but in a process running as tedge user, which fails. This should not happen.
   • this codepath worked back when tedge-agent ran as root by default, but now it runs as tedge and privileged operations should be handled by tedge-write. Unfortunately we missed a path where tedge-agent attempted to chown directly
   • the test suite was missing a case where a file was owned by root

Second try succeeds because:

• the logic for config update is to set ownership if creating a new file, but preserve existing ownership if the file already exists
• unnecessarily, the same behaviour was applied to temporary files; once the file was there we skipped setting the permissions and subsequently moved the file out to the destination. Once the temporary file did not exist any more, the cycle would repeat.

What should happen:

• [x] test suite should be updated to cover this missing case and check we're not missing any others (#3148)
• [x] avoid setting permissions directly in config-manager and other callers, unless we're sure we have necessary privileges (#3148)
• [ ] make sure we're not leaving any temporary files (not necessary for the resolution of this issue, will be done in a follow-up PR)

[gligorisaev]

Bug is not reproducable anymore, runned also the flakie test finder 100 times, no error detected