Ideas for Settings & Preferences

[tlambert03]

🚀A list of things that might benefit from persistence

Following up on this https://github.com/napari/napari/pull/834#issuecomment-590112950, this issue is a place list ideas for things that might make good settings/preferences (once we have a persistence layer in place). Feel free to edit this comment directly, or start your own lists.

• [x] theme (superseding #943)
• [ ] monitor DPI (#820)
• [x] call order of plugins (after #937)
• [x] window position/geometry
• [ ] default colormaps or color combinations (as discussed in #619)
• [ ] magic-naming for layer (off by default in #1008)
• [ ] PySide2 vs PyQt5 preference (if both installed)
• [x] highlight thickness for shapes / points layers
• [x] dask cache size maximum, and dask fusion settings (https://github.com/napari/napari/pull/1173#issuecomment-620231964)
• [ ] opt in for telemetry
• [ ] (need to know if it's the first launch)
• [ ] version updates (has user been asked)

[GenevieveBuckley]

Font size in the integrated ipython console.

I would really like to be able to change this, I keep going to use the integrated ipython console & then remembering this is the reason I don't like using it. And yes, I do have terrible eyesight.

[jni]

I just came across omegaconf. Not that we need any more choices for napari config management, but I think part of the reason we haven't got this done is that nothing felt "satisfying", and OmegaConf kinda does! See the "creating" part of the user guide, lots of nice stuff in there.

It is a young project, which might be reason enough to avoid it, but I still thought it was worth considering.

[tlambert03]

looks nice! just went through their binder tutorial which is great. For me, i feel like the biggest conflict wasn't necessarily the implementation or the API (I do like what I see in omegaconf, but I wasn't finding the dask approach particularly lacking either)... it was more the conflict between the concept of app settings and library settings, as described in https://github.com/napari/napari/pull/1183#issuecomment-694243607. I feel like if we just said "sure, let's have some sort of private persistence layer that the app can write to, and the user doesn't need to touch" ... we'd be off and running. I don't think it's impossible to do both, but it's definitely the harder challenge and we're biting off a lot.

[Carreau]

Hi all, from yesterday productive discussion about setting a couple of my mental notes

• Config can also be useful for remembering custom keyboard shortcuts.
• It would be nice to be based on a json-schema or Json-LD approach to autogenerate GUI to set config, potentially with magic gui.

• call order of plugins (after #937)

This might not be sufficient if two plugins implement multiple readers the overlaps and you want plugin A for filetype X and plugin B for filetype Y

• You may want to have system wide config that prevent users from overwriting them.
• Yaml has many issue especially when human-written; – you are aware of the safe_load vs load from the codebase, but many values in Yaml can be ambiguous (no abbreviation for norway for example is interpreted as false, any 0-leading number will be interpreted as octal, other formats are interpreted as dates...) so I would be worried about yaml.

Font size in the integrated ipython console.

IPython config can be in json as well, so a first hack cold be to just pass a given config section through.

[tlambert03]

This might not be sufficient if two plugins implement multiple readers the overlaps and you want plugin A for filetype X and plugin B for filetype Y

Still thinking about this... but there might be some semantics that are important here. I think that if a single package wants to implement one of our hookspecs (like napari_get_reader) multiple times, then they will technically need to use two entry points, each of which gets assigned a key in our plugin manager. So "plugin" in this case refers more to the entry point than the package on the whole. Putting aside for the moment the fact that the terminology is a bit confusing, that may mean that this isn't actually a problem (since you can sort Plugin A filetype X reader independently of Plugin A filetype Y reader). On the other hand, if the package has decided to implement only a single napari_get_reader and hidden the file selection logic from us, then indeed... we'd be out of luck with regards to sorting.

Let me know if that was already how you were seeing it, or if that changes things slightly

[tlambert03]

Putting aside for the moment the fact that the terminology is a bit confusing

"un-putting" this aside... this arrangement is definitely up for modification! Happy to chat more about it

[Carreau]

Yes, and I think we brought that up yesterday that one plugin might have many readers and you may want to change reader. I think that ordering cover most of the cases whether we want to change that in config or by recommendation in how to write plugins, or something else can be worked on; I'm not yet familiar enough with the types of plugin to give you any advice on this.

[sofroniewn]

We might want to think about this in the context of #2114 too (that discussion been more focused on GUI plugins, but reader plugins might interesting to consider too).

[goanpeca]

Hello, had a meeting today with Pam, to go over some of the Qt issues and spend some time on the settings one to do some small brainstorming session. The more I think about it, the more I keep prefering Json Schema to define how settings are defined by core and plugin.

Using pydantic, omegaconf, others, keep tying the system to python and specific implementations, where using JSON schema proivdes a standard, language independent (there are many json and jsonschema checkers on different languages) way of providing options.

We have different moving parts here:

1. Defining what an option is and the information it requires.
2. If we use a Json schema, then we get checking and validation "for free"
3. Pass this information to a UI builder that will turn a bunch of schema files (1 per plugin?) so that a preferences dialog can be built (in this case a Qt one). Something like this could be used, or serve as base for inspiration https://github.com/agoose77/qt-jsonschema-form, and there are surely others.
4. Have this UI return a dict/json to whatever system will be in charge of reading/writing configuration options.
5. Decide on the persistence layer and the format. The thing in charge of reading/writing configuration.
6. Although JSON could be used here (and keep all json based), it is probably not the best format because it is verbose and does not allow for comments. I suggest exploring YAML and TOML, even if they require a new dependency.

Since not all of these things need to be decided at once, we could split this issue into several smaller ones and start tackling them separately.

[sofroniewn]

hi @goanpeca, thanks for this great write up!

Using pydantic, omegaconf, others, keep tying the system to python and specific implementations, where using JSON schema proivdes a standard, language independent (there are many json and jsonschema checkers on different languages) way of providing options.

I like this! Language independent would be nice especially if we one day wanted a web based way to change settings etc

Since not all of these things need to be decided at once, we could split this issue into several smaller ones and start tackling them separately.

This is great! Yes, I encourage you to do so. I very much like thinking about it that way. Something that @tlambert03 brought up yesterday that would be good to get your perspective on are which of these choices do we need to decide now vs which of these choices could we decide later or if we changed our mind on then it wouldn't be that big a deal to change later. That would help me know where to focus my input.

Broadly speaking though I like this direction and am very open to JSON Schema, though I have never worked with it myself. Curious @tlambert03 @jni @justinelarsen or others if you have thoughts?

[justinelarsen]

I'm open to both YAML or JSON. I've used JSON much more than YAML, coming from a web background. I think either would make sense for storing the settings/preferences data. YAML seems to be widely used for configurations - I know Docker uses it.

Some important considerations:

• Comments: Like @goanpeca mentioned, JSON doesn't support comments. So if this is a must, go with YAML.
• Web-based interface: If a web-based interface will be developed some day and we want to share the configurations or the implementation, parsing the JSON will be easier.
• Syntax: If users will edit the files, Python users will likely prefer YAML due to the indentation. It is also more human readable. If we don't expect users to edit the files, YAML's easier readability may actually be a negative as it may cause longer processing times for (de)serialization.
• Complex structures: YAML supports serializing complex structures with references (also contributing to longer processing times). Will we need to use complex data structures?

This is a good paper that compares JSON/YAML for data serialization: https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.1048.2508&rep=rep1&type=pdf

[alisterburt]

we chatted a little bit about the need to store mouse bindings and key bindings in the settings in the last euro dev sync meeting - just adding it here so it's kept somewhere!

[neuromusic]

Adding to the list of "things that might make good settings/preferences"...

Plugin developers may wish to offer their own plugin-specific "settings and preferences" for users of the plugin. Might it be possible to use this settings infrastructure to...

• store plugin-specific settings (through namespacing?)
• let a plugin developer define the settings & defaults (through a datastore or similar abstraction?)
• let end users manipulate plugin settings (through plugin tabs or sections in the settings UI?)

[adamltyson]

Adding to the list of "things that might make good settings/preferences"...

Plugin developers may wish to offer their own plugin-specific "settings and preferences" for users of the plugin. Might it be possible to use this settings infrastructure to...

* store plugin-specific settings (through namespacing?)

* let a plugin developer define the settings & defaults (through a datastore or similar abstraction?)

* let end users manipulate plugin settings (through plugin tabs or sections in the settings UI?)

+1 for this. I'd like all of these features for the cellfinder plugin. Two things would be particularly helpful:

• (Optionally) save the plugin state - cellfinder will have a lot of variables that once optimised will not need to be touched
• Remember files chosen through the UI (e.g. trained models). Ideally these could be given "nicknames" so a user can choose Trained neuron model rather than ~/.cellfinder/models/model-epoch.10-loss-0.059.h5 etc.

Flexibility of the implementation of these persistent settings is important, but I have no strong feelings about the specific implementation.

[neuromusic]

Another consideration for YAML vs JSON is whether the napari community will want to have hidden/obfuscated settings for experimental or pre-release features, where you don't yet want to enable configuration through the GUI, but do want to let a proficient user manually tweak the configuration.

For this kind of use case, YAML will probably be better than JSON due to the readability & more forgiving syntax.

[Carreau]

due to the readability & more forgiving syntax.

I find that json with indent is not that much harder to edit when you don't have deeply nested structure. The forgiving syntax of yaml has though been quite problematic for me when it is wrong.

If each plugin get its configuration file, then you likely want a version info in each plugin config so that when a plugin is updated it has the opportunity also upgrade its setting file.

IMHO saving plugin "State" should not be conflated with setting. It is related; but can probably have widely different requirements on the readability and whether the "state" can be thrown away if invalid. I can also imagine state being stored in a binary format that hold more complex data format than what json can do.

[goanpeca]

Thanks for all the input! Now geting back to this work :)

I will be splitting different topis discuss here into more fine grained issues. I will try to copy paste comments from this thread into the new issues.

@Carreau point on state and setting is very important and wort more discussion in a more specific issue as well :)