Configuration configurations + getting started

[Nick-Mazuk]

Is your feature request related to a problem? Please describe.

There are three problems here which can likely be solved very similarly.

1. Users have no clue what parameters are configurable
2. Users don't know how to configure parameters
3. Users may not know what a script does/how to use it based solely on it's title

I was talking with @jwink75 earlier and I think we have a rough solution to solve these.

Describe the solution you'd like

If we had a "startup dialog" that only ran the first time a script runs, we can share this information with the user.

Startup dialog

In the case of "startup dialog", we can simply modify the original .lua script file with a variable saying "I've seen this before". This way, if a user updates a script, they still see the startup dialog again just in case something has changed.

For this, I'd recommend taking the finaleplugin.Notes value (which is a multiline string) and using that as the startup text. I can then use the build scripts to add all the necessary code to show the startup dialog. That way, we don't need to implement it every time. Alternatively, we could create a library function for this, though I'm not sure how it would impact development. And finaleplugin.Notes is useful because that's already displayed on the website (just no one uses it). Using finaleplugin.Notes would both help documentation, search, and user experience without needing a ton more work for each developer.

Configuration

In the case of configuration, if the config file does not exist, running the configuration script creates one and shares with the user how to configure it (perhaps even presenting the available options?). This has a number of advantages.

Firstly, users would then know they can configure the scripts.

Secondly, if the script originally writes the values into a file, then we can write a script to lookup the potential values for the configuration. For instance, we can then create a generic "Settings" script that looks through jwpluginstorage.xml to find all Lua scripts installed, check to see which ones have config files, then let users update any of those configurations.

Disadvantages of auto-creating configs

I see two disadvantages with this approach.

1. Dynamic configuration values are inherently possible. We'd need to update the config syntax to include which values are the default and not changed. If a value is marked as the default, then we can safely use dynamic configuration. There are a few more edge cases, but we can work those out later.
2. If a user updates a script, we cannot guarantee all configuration values are present in the config. This is solvable. I'd recommend that we both inform the user they can configure more values as well as updating the config with the new default values.

Additional context

By the way, do we need to name the configuration file manually as developers? It seems like none of us are, in which case we can use finale.FCString():SetRunningLuaFilePath() .. ".config.txt" or similar instead of providing the configuration file name.

@rpatters1, @jwink75—thoughts?

[rpatters1]

Startup dialogs are barriers. I absolutely hate startup dialogs, even once. They are like EULAs. Everyone just curses them and clicks thru without reading. This is why, for example, rather than put up some stupid error message about how the script can't run without a selection, I try to design scripts so that they can run without a selection. Obviously there are exceptions where an error message is inevitable.

As for automatically creating the config, the main issue I see with it is for values that are initialized dynamically and overridden. This is brought up in the "disadvantages" section. But let me clear: the configuration function already has a way to do this. You simply omit the config values you aren't interested in overriding, and the defaults from the script are used instead. Issue 2 is not a disadvantage. It is a designed-in feature that addresses issue 1. The config file is completely optional, and every field in it is optional.

My preference would be, if we are automatically creating config files, to create them with every field commented out by default. That way the file would be there but the script would still retain its default behavior unless the user edited it. Auto-creation could also be easily added to the configuration function.

I quite like the idea of automatically generating the file name from the running path, assuming we can extract the filename by itself without the .lua extension (which I imagine is fairly simple). The only concern I have is that some of the libraries also have config files, so we need to be a little careful about our testing to be sure the library config files get the right value.

Another approach to configuration would be to create a Configuration script that knows about all the config files and config lists in scripts. If we went this route, I would want an automated way for the script to find all the configurable parameters in each of the .lua files. To me the requirements of configuration are:

• Config is optional, and every field is optional.
• Script writers should be able to design configurable parameters without any dependencies on other scripts.
• A default value for a field should have the option of being dynamic (based on the document or other factors). Then the config overrides it only if that field is in the config file.

Ultimately, though, I'm not sure why we need expend a lot of effort to hold users' hands. Configuration is a tool for power users. People who have no experience with text editors should probably be taking the defaults. If we auto-create the files (with the fields there but commented out), we can point to a URL that shows how to change them. I'm a firm believer in forcing people to learn how to fish rather than handing them a fish. That's what JW Lua is about.

[rpatters1]

I should probably add my process for using config file (since I invented it, lol).

• I have added a local git repo to my script_settings directory to track changes. (For this reason, I am not keen on a lot of automatic mucking around with it by lua scripts.)
• I frequently work with a text editor window of the config file open next to my Finale document. This allows me to change config values on the fly as I work.

The whole reason I added the configuration library is that directly editing the script files could cause havoc with this repository. Now that we've gone to a distribution directory, it may be that many users would rather edit the scripts directly. The question arises as to whether configuration is a user-targeted tool or whether it is more of a developer-targeted tool.

[jwink75]

Ha! This same thought occurred to me after your last post… if the target is power users, why not keep everything within the script? I know I’m not always good about putting all the variables up front, but maybe as a recommended practice all variables could be set up in one chunk of code (variable = finale.FCSomeFeature() ), and then there could be another clearly labeled section where the actual values live for easyish customization later. Does that make sense?

Sent from my iPhone

On Sep 1, 2021, at 6:42 AM, rpatters1 @.***> wrote:

 I should probably add my process for using config file (since I invented it, lol).

I have added a local git repo to my script_settings directory to track changes. (For this reason, I am not keen on a lot of automatic mucking around with it by lua scripts. I frequently work with a text editor window of the config file open next to my Finale document. This allows me to change config values on the fly as I work. The whole reason I added the configuration library is that directly editing the script files could cause havoc with this repository. Now that we've gone to a distribution directory, it may be that many users would rather edit the scripts directly. The question arises as to whether configuration is a user-targeted tool or whether it is more of a developer-targeted tool.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

[rpatters1]

Well, we need configuration as developers. I don't use the deployment scripts. I run them straight out of src. It's really the only way to work on them.

I've already followed a de-facto standard, at the top of the script.

-- Block 1
config {
   data_value1 = 1,
   data_value2 = 2
   --etc
}

-- Block 2
config.data_value2 = finale.FCSomeFeature()

-- Block 3
configuration.read_config(file_name, config) -- or maybe those parameters are reversed, I can't remember

[rpatters1]

Also, an actual power user is going to prefer a config file. That's because if they edit the distro script, they will lose their changes if they refresh the script to the latest version.

[Nick-Mazuk]

My preference would be, if we are automatically creating config files, to create them with every field commented out by default. That way the file would be there but the script would still retain its default behavior unless the user edited it. Auto-creation could also be easily added to the configuration function.

I like this approach. By commenting out the defaults (instead of simply omitting them), we can still parse them later on to figure out what options are configurable if we want to.

I quite like the idea of automatically generating the file name from the running path, assuming we can extract the filename by itself without the .lua extension (which I imagine is fairly simple). The only concern I have is that some of the libraries also have config files, so we need to be a little careful about our testing to be sure the library config files get the right value.

I believe it's simple to remove the .lua. Most languages have a "replace substring" function where we can just using file_name.replace(".lua", "").

Good point about the library files. Perhaps instead of completely omitting the file name argument, we can make it optional. Then the libraries can still supply their own file name. That way, we can avoid name collisions while still making it simpler to use.

Another approach to configuration would be to create a Configuration script that knows about all the config files and config lists in scripts. If we went this route, I would want an automated way for the script to find all the configurable parameters in each of the .lua files

I do like this approach. I think parsing the configurable parameters from the Lua scripts themselves could be very difficult. Parsing a config file would be much simpler (since @rpatters1 designed them to be parsed).

I do agree with the requirements you listed are desirable.

Since Finale is supposedly going to make an announcement sometime about JW plugins (and perhaps JW Lua), I'd lean towards waiting until the announcement before making the configuration script. It's likely not a good idea to invest a ton of time in a feature that, while nice, is at best only for powerusers and at worst obsolete if Finale is not compatible with JW Lua.

Ultimately, though, I'm not sure why we need expend a lot of effort to hold users' hands. Configuration is a tool for power users. People who have no experience with text editors should probably be taking the defaults. If we auto-create the files (with the fields there but commented out), we can point to a URL that shows how to change them. I'm a firm believer in forcing people to learn how to fish rather than handing them a fish. That's what JW Lua is about.

I'm of the opinion that every Finale user should be able to use JW Lua and the JW Lua scripts we create without much hassle. I'm also of the opinion that the defaults should be great such that 95% of uses never need to mess with configuration. I do not believe that we should be forcing people to learn to code just to use JW Lua scripts.

But, if someone needs to configure the scripts, they will probably figure it out. We are giving out these scripts for free, so we're not under any obligation to provide a world-class experience, so I don't think it needs to be easy to configure (easy is nice, though). Additionally, for 95% of people (just a guess) configuring the scripts will probably lead to worse results for them.

However, I think the main problem right now is that of the hundreds (if not thousands) of people that have downloaded these scripts, I'd guess that less than 5 even know that they are configurable. So even if someone wanted to, needed to, and was willing to learn how to configure the scripts, they wouldn't even know to look. And while I much prefer to teach people to fish over handing them sushi, I don't think these powerusers should have to learn to read the source code just to configure their scripts.

[Nick-Mazuk]

if the target is power users, why not keep everything within the script?

That's a good thought, but I think the way we currently use the configuration is probably smoother (for the reasons Robert mentioned). It is how almost every piece of software with settings/configuration works.

[rpatters1]

As for JW Lua compatibility with Finale 27 (and beyond), there is no reason to think that there will be an ARM version of Finale any time soon. And even if there is, x86 Windows doesn't seem like it's going anywhere. (That said, I suspect that if Finale survives long enough as a platform, there will be a WinFin version for ARM. Just no time soon.) So JW Lua will still be compatible with WinFin for the foreseeable future.

As for Mac, I suspect users will still be able to select running in x86 mode if they want to use JW plugins, esp. JW Lua. It is obviously not an ideal setup and can't last long term. But it may allow Mac users to keep using JW Lua for a year or two more anyway, if they are willing to run that way.

Ultimately, someone has to convince Jari to release the JW Lua source. If Makemusic can't (or isn't interested in it), someone else must. That should be our priority. Surely with the Facebook and email groups there is someone who can reach him. I have tried myself without success, though.

[rpatters1]

Nick: I sent you a follow up on Facebook Messenger. It may be that you'll have to go to the Message Request area to see it. Anyway, I'm just posting here to let you know.

[Nick-Mazuk]

Nick: I sent you a follow up on Facebook Messenger. It may be that you'll have to go to the Message Request area to see it. Anyway, I'm just posting here to let you know.

Got it, I'll look at that later today.

[ThistleSifter]

I think that the sanest way of handling config would be through JW Lua itself by allowing plugins to register whether or not they need a configuration file (or more than one). For each required config, the plugin would then register a config definition (fields, their types and ranges of values) and a set of default values. JW Lua could then, through its UI, have a window or tab where users can edit configurations, save them to file or load different ones. For plugins with their own complex or custom configuration requirements, the plugin could instead register functions that would display the editing window and validate existing configuration files when loaded.

[rpatters1]

If we can modify JW Lua, that might make sense. It already has a parameters feature, but I find it cumbersome and not very useful. If one can't use parameters to create multiple menu items for the same script, I'm not sure what the point of them is. What I imagine is a simple header script users could insert that would be added to the Lua context before the main script is called.

But those bridges will only be crossed if JW Lua has a new version.

On Wed, Nov 3, 2021 at 9:28 PM ThistleSifter @.***> wrote:

I think that the sanest way of handling config would be through JW Lua itself by allowing plugins to register whether or not they need a configuration file (or more than one). For each required config, the plugin would then register a config definition (fields, their types and ranges of values) and a set of default values. JW Lua could then, through its UI, have a window or tab where users can edit configurations, save them to file or load different ones. For plugins with their own complex or custom configuration requirements, the plugin could instead register functions that would display the editing window and validate existing configuration files when loaded.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Nick-Mazuk/jw-lua-scripts/issues/109#issuecomment-960383660, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACYB6XYXH2YMY33YBDKTVETUKHVR7ANCNFSM5DFBBZ4A . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.