Configuration value disappearing on Vault restart

[Xteuk]

When creating a VaultApplication that inherits the `MFiles.VAF.Extensions.ConfigurableVaultApplicationBase', the following behaviour occurs:

• If a Configuration property is defined with a value that's the default value, that property disappears on Vault restart. (Whether it's defined by name, alias id, etc.).

Why is this a problem? Even though the VAF is working effectively (using the default value)

• Mainly: M-Files Admin displays a local error "A value is required" and this message cannot be fixed by configuration or development** (if the value is mandatory, i.e. not marked "AllowEmpty").

• There is no way to distinguish between a default value and a real null value.

• There is no way to quickly "fix" a VAF via configuration if (for example) there is a bad configuration like this

  [DataMember, MFClass, JsonConfEditor(DefaultValue = "CL.MyClass")]
  public MFIdentifier MyClass; // dev forgot to actually put the default value here

Note: If the VAF extends the default `MFiles.VAF.Core.ConfigurableVaultApplicationBase' then this behaviour disappears completely.

Example:

    [DataContract]
    public class Configuration
    {
        [DataMember, MFClass, JsonConfEditor(DefaultValue = 0)]
        public MFIdentifier Test1 = 0;
    }

and configuration

{
    "Test1": "0"
}

After reboot:

{ }

[CraigHawker]

Firstly: excellent issue formatting and description, thank you!

This is currently by design; the VAF extensions has the ability to control the upgrade process when the configuration itself is changed. A side-effect of this is that, without removal of default values, the configuration would balloon to include all possible values regardless of whether they were created explicitly or not. If you do want to turn off the configuration upgrading process then it's documented here: https://github.com/M-Files/VAF.Extensions.Community/tree/master/MFiles.VAF.Extensions/Configuration/Upgrading#disabling-configuration-upgrading (simply override a method and return null).

I would like to know more about some of these situations though (please- I really do want to know more about the use-cases here; I am not simply dismissing them!):

Mainly: **M-Files Admin displays a local error "A value is required" and this message cannot be fixed by configuration or development (if the value is mandatory, i.e. not marked "AllowEmpty"). Why can "AllowEmpty" not be set here? Can you give a practical example as to why this would happen?

There is no way to distinguish between a default value and a real null value. At a practical, runtime, level, how would you know anyway? Acknowledging you simply get the re-hydrated object instance (and not the raw JSON), what's the practical difference between a value of zero set implicitly when the object is created and one that is explicitly set? I understand there's a conceptual difference, I'm asking why it's important at a practical level.

There is no way to quickly "fix" a VAF via configuration if (for example) there is a bad configuration Yes, I can see that, although I argue that the solution is to fix that member.

By the way: there is one situation where this definitely does cause an issue, but it's not one of the ones above.

[Xteuk]

Wow, that was a quick answer, thank you.

To provide more information : This became an issue when, after upgrading dependencies for some existing VAFs (one with huge configuration), errors started to pop up in the M-Files Admin.

After lots of testing, I've just this morning understood what was causing it AND that the VAF behaviour was actually not endangered. So, to be honest, part of the issue was the time needed before having an answer to provide.

Part of the issue, too, is that the customer started to notice disappearing values in the configuration, and asked us to fix that (without me being able to answer).

I'm soon look at the information you provided, to better understand the pros and cons. I'm pretty confident now that it'll be ok.

That said :

About "AllowEmpty" : I'm not entirely sure yet if "AllowEmpty" is the good one, because i'm still struggling between "AllowEmpty=true", "Required=false" and "JsonConfEditor(IsRequired=false)". I had issues hiding some of those errors in the customer's VAF, so it was an 80% fix. I will continue my testing and, if i find a 100% fix, i'll stick to it.

Default value: Practically speaking, a default value IS a value, whereas a null value is, well, "== null". I've used defaults to give customers the least amount of configuration, but the most options. I've used the manual null value to inhibit behaviour. Now i'll recon it's a design flaw on my part.

Quick fix: I agree with you that the best fix is on the member itself, in the VAF. Though the most efficient one, it's also the slowest one. When trying to bring a prod quickfix while sharing desktop and voice with the customer, it's ... annoying.

Now that i know why and how this is happening, i can manage. Thank you for the answer.

Oh: maybe having a hint somewhere, like in the log, on what's causing configuration modification, could be a good help to people investigating.

We do have other disappearing values for another customer, but they're not default values so it's different. I won't develop here, but i'll use what you learned me.

Have a nice day,

Cyril

[CraigHawker]

I definitely take some of your points onboard. I think the below should happen:

1. We should have better documentation around AllowEmpty/Required/etc. If you have specific examples which have tripped you up then please contact me via email (firstname.lastname at the company domain, which I am sure you know).
2. The automatic upgrade process should use the logging framework to log that it ran, maybe with a link. It cannot state what exactly was dropped, but it can state that it ran which may be good enough for people to at least query what it is.

I will try to get these done in Q1.

In addition, please keep me updated around the other values. Let's see what's happening there, either via here or via other mechanisms. Also note that we have a forum where you can comment if "issues" here aren't appropriate: https://community.m-files.com/groups-1602070992/developers/

[CraigHawker]

A quick note that, after some discussion internally, we are proposing that the configuration upgrading becomes opt-in. The PR for this change is here: https://github.com/M-Files/VAF.Extensions.Community/pull/123, and a preview release containing these changes is in nuget as 23.12.0.3-confupgrade-optin.

We will open up feedback on this internally and externally. Any change in the full release will not happen until 2024 now.

The primary concern I have is that if people are relying upon this then there is no "breaking" or visible change that will inform them that the new version disables the functionality, which could be an awful experience.

[Xteuk]

Thanks for letting me know