Improve downgrade YAML handling

3djc commented 2 months ago

Is there an existing issue for this feature request?

[X] I have searched the existing issues

Is your feature request related to a problem?

Quoting @rotorman on Discord: "Numerous users seem to downgrade their radio, e.g. while trying out a release candidate version and then returning to latest (lower) release. The result is more than often a corrupted or unusable YAML files. I am thinking if it would be beneficial to give users a warning if a newer YAML version is detected on start than what current firmware expects. "

Describe the solution you'd like

A couple of alternatives:

warning at start (imho, useless, users will ignore it)
warning at start, backup of newer version files, try to work with those
warning at start, backup of newer version files, work with default
alert and refuse to start

Describe alternatives you've considered

No response

Additional context

No response

pfeerick commented 2 months ago

Another alternative, could be to show on start (which will only occur once since version would be overwritten on shutdown) and on individual model load if version number newer than firmware, just like Companion does. It is not useless, as yes can ignore, but cannot complain they did not know it would not have broken settings which is the basis for current complaints.

IMO, not much more is warranted... I would think it is a really only a handful of people vs entire userbase - we just think it's a lot as we generally only see the negative comments.

Another alternative is to do an OpenTx and delete all models and settings. Although here we also have to option to move the model settings to a folder, forcing user to do something about it.

On Wed, 8 May 2024, 2:28 am 3djc, @.***> wrote:

Is there an existing issue for this feature request?

I have searched the existing issues

Is your feature request related to a problem?

Quoting @rotorman https://github.com/rotorman on Discord: "Numerous users seem to downgrade their radio, e.g. while trying out a release candidate version and then returning to latest (lower) release. The result is more than often a corrupted or unusable YAML files. I am thinking if it would be beneficial to give users a warning if a newer YAML version is detected on start than what current firmware expects. " Describe the solution you'd like

A couple of alternatives:

warning at start (imho, useless, users will ignore it)

warning at start, backup of newer version files, try to work with those

warning at start, backup of newer version files, work with default

alert and refuse to start

Describe alternatives you've considered

No response Additional context

No response

— Reply to this email directly, view it on GitHub https://github.com/EdgeTX/edgetx/issues/4982, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABJ66KOZWCXWXWJTEFFWHJTZBD6KBAVCNFSM6AAAAABHLK3B7KVHI2DSMVQWIX3LMV43ASLTON2WKOZSGI4DGNZXGU2TSMA . You are receiving this because you are subscribed to this thread.Message ID: @.***>

3djc commented 2 months ago

Actually, I have not seen complaints, just people with broken setups , not a few by any means

philmoz commented 2 months ago

What about on startup, if any of the radio or model files are an older version, then create a backup folder with date/time in the folder name and copy the models and radio yaml files there. That way if the user downgrades they have an automatic backup to use.

3djc commented 2 months ago

That would be my proposal 3

philmoz commented 2 months ago

I interpreted your number 3 as backing up newer version files when a downgrade was detected and not using them.

I am suggesting backing up older version files when an upgrade is detected so the user can later manually downgrade. I.E. on startup, if the radio or current model version is older than the firmware open an 'Upgrading' page which backups up the old version MODELS and RADIO folders (telling the user where they are backed up to), then upgrades all the model files and radio yaml file.

philmoz commented 2 months ago

I interpreted your number 3 as backing up newer version files when a downgrade was detected and not using them.

I am suggesting backing up older version files when an upgrade is detected so the user can later manually downgrade. I.E. on startup, if the radio or current model version is older than the firmware open an 'Upgrading' page which backups up the old version MODELS and RADIO folders (telling the user where they are backed up to), then upgrades all the model files and radio yaml file.

Once the ability to backup models/radio to a date stamped folder is in place it could be exposed to the user to allow backups to be manually created at any time.

pfeerick commented 2 months ago

There have several over the years, around each release where there have been changes to data model.

(Re-)established and/or fixed (backup 'n restore)... missing on colorlcd, possibly broken in B&W (probably an issue for that)... So option two then? Backup and attempt to continue?

On Wed, 8 May 2024, 7:48 am philmoz, @.***> wrote:

I interpreted your number 3 as backing up newer version files when a downgrade was detected and not using them.

I am suggesting backing up older version files when an upgrade is detected so the user can later manually downgrade. I.E. on startup, if the radio or current model version is older than the firmware open an 'Upgrading' page which backups up the old version MODELS and RADIO folders (telling the user where they are backed up to), then upgrades all the model files and radio yaml file.

Once the ability to backup models/radio to a date stamped folder is in place it could be exposed to the user to allow backups to be manually created at any time.

— Reply to this email directly, view it on GitHub https://github.com/EdgeTX/edgetx/issues/4982#issuecomment-2099368147, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABJ66KM6E5CZ4JXTZOSO6G3ZBFD2VAVCNFSM6AAAAABHLK3B7KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOJZGM3DQMJUG4 . You are receiving this because you commented.Message ID: @.***>

philmoz commented 2 months ago

If an automatic backup was made when the user upgraded their firmware version, and they were told where the backup was saved, then why do we need to do anything on a downgrade (other than maybe showing a warning)?

pfeerick commented 2 months ago

Huh? I'm confused now. So you instead want to show a notice when upgrading that a backup is saved, and not show anything if they downgrade? That is backwards, since the majority upgrade, and the minority downgrade, and will lead to it being ignored. Or are you saying we should start keeping copies of older versions of settings on each upgrade, which may lead to a complete mess in the future?

If it is more involved than a "you are running a newer version than the model file" warning, the I think the process should be

warning at start (when the downgrade attempt/conflict is detected), backup of newer (than current) version files, try to work with ~~those~~ them. You are not working on the backup, you are still working with the files in MODELS/RADIO, but you have a backup.

philmoz commented 2 months ago

I'm not against doing something when a downgrade is detected; but making a backup of the newer files at that point is a bit like shutting the barn door after the horse has bolted.

I think most users will upgrade to a new release and not look back. The users we need to try and help are the ones who upgrade without making a backup, then want to roll back. Despite all the warnings in the release notes this is going to happen.

The only guaranteed safe way to roll back is to restore a backup.

By doing an automatic backup on upgrade, and telling the user about it, we are being pro-active and giving all users a safe way to roll back.

pfeerick commented 2 months ago

Ok, so are you thinking "if major and/or minor version - not patch version - later than current major and/or minor version - backup the settings - show progress while copying files (ie. how conversion was shown)"? If, firstly, the backup/restore functionality is restored to proper functionality across the board, it would then accessible as another backup entry. That makes sense.

In that case, I think it would be fine to just have a "you are using newer settings with older firmware" nag message on startup/model select, and leave resolution of that self-created problem to the user.

philmoz commented 2 months ago

Yes that's what I'm thinking:

if firmware major.minor > radio.yaml major.minor

Show 'Upgrading...' page
Backup MODELS and RADIO to BK_xxx_YYYYMMDDHHMM folder with progress bar (xxx is radio.yml major.minor ver)
Upgrade radio.yaml and all modelXX.yaml files (with progress bar)
Show user result and name of backup folder.

3djc commented 2 months ago

I interpreted your number 3 as backing up newer version files when a downgrade was detected and not using them.

I am suggesting backing up older version files when an upgrade is detected so the user can later manually downgrade. I.E. on startup, if the radio or current model version is older than the firmware open an 'Upgrading' page which backups up the old version MODELS and RADIO folders (telling the user where they are backed up to), then upgrades all the model files and radio yaml file.

True, but both scenario are actually valid. You could be using say 2.10 for several week/month, encounter something odd ro simply different. "Hum, I'm sure it worked before, I'll install my previous (major) version to check..."

Your backup made a while ago won't really help you there

philmoz commented 2 months ago

I interpreted your number 3 as backing up newer version files when a downgrade was detected and not using them. I am suggesting backing up older version files when an upgrade is detected so the user can later manually downgrade. I.E. on startup, if the radio or current model version is older than the firmware open an 'Upgrading' page which backups up the old version MODELS and RADIO folders (telling the user where they are backed up to), then upgrades all the model files and radio yaml file.

True, but both scenario are actually valid. You could be using say 2.10 for several week/month, encounter something odd ro simply different. "Hum, I'm sure it worked before, I'll install my previous (major) version to check..."

Your backup made a while ago won't really help you there

I agree; but we can never account for every possible scenario. And an old backup is better than no backup.

I also see what you are saying - in this case it might be useful to have the upgraded / newer files backed up as well.

ThomasKuehne commented 2 months ago

A) version change triggers backup on loading a yaml: offer to make a backup if (version in YAML != firmware version).

That covers the common upgrade and downgrade scenarios. However that might not be enough 1) when firmware version is the same but certain compliation options changed 2) nigthly builds and selfbuild might need some extra care

How about an additional YAML compability indicator? e.g. based on a checksum of all names, data types and sizes of all other YAML fields.

B) minimize update impact after loading a yaml: track if the the user or system makes any content changes on shutdow: write only those model yamls where the tracking indicates content changes

EdgeTX / edgetx