Closed NathanW2 closed 4 years ago
Agreed. Very good improvement. Thanks Nathan.
All profiles would live in a single top level folder, maybe in ~, %HOME% to make it easy for the admins and users to find if needed. A link the about screen would also have the option to locate the current profile folder.
I think we should respect the standard folder locations here. So Appdata\Roaming (or local?) for windows, and ~/.config/qgis for linux. Not sure what the OSX standard is.
We've been abusing people's home folders with the .qgis2 folder for a long time, it'd be good to start following the standard now that we've got the chance.
See also http://hub.qgis.org/issues/11926
At the same time we could also should split up config and cache folders, so users aren't unnecessarily backing up temporary files when they are saving their qgis settings.
Just so I understand this fully - this means no more registry use on windows?
Yes, no more registry on Windows. A single uniform setup for all platforms.
Happy to follow the standard folder layout.
On Fri, Nov 25, 2016 at 9:53 AM, Nyall Dawson notifications@github.com wrote:
Just so I understand this fully - this means no more registry use on windows?
— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/qgis/QGIS-Enhancement-Proposals/issues/82#issuecomment-262857048, or mute the thread https://github.com/notifications/unsubscribe-auth/AAXS3OGe43Ziv35w8LXgqEsT1mzwBBdMks5rBiOGgaJpZM4KwZyI .
If we provide a way to jump to the folder from inside QGIS it doesn't really matter the location it's saved. So happy to match what the standard is for each platform.
When would someone need to jump to this folder? For backups or something?
Yeah, just generally I don't think it's a good idea to not have a quick way of getting there. Of course, people really shouldn't but just in case.
We could also provide UI for exporting/importing, etc
On Fri, Nov 25, 2016 at 10:23 AM, Nyall Dawson notifications@github.com wrote:
When would someone need to jump to this folder? For backups or something?
— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/qgis/QGIS-Enhancement-Proposals/issues/82#issuecomment-262859074, or mute the thread https://github.com/notifications/unsubscribe-auth/AAXS3CeD9i8ZencnnQsWDu-RKN8417xWks5rBiqYgaJpZM4KwZyI .
+1 for getting rid of win registry +1 for a standard location unsure about designing our way to manage the config folder: what other major sw do? Can't we leave this to the OS file manager?
Yes, I just mean have a way to open that folder e.g this works cross platform QDesktopServices::openUrl(QUrl::fromLocalFile("path to folder")) that will open that folder using the OS manager.
On Fri, Nov 25, 2016 at 4:56 PM, Paolo Cavallini notifications@github.com wrote:
+1 for getting rid of win registry +1 for a standard location unsure about designing our way to manage the config folder: what other major sw do? Can't we leave this to the OS file manager?
— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/qgis/QGIS-Enhancement-Proposals/issues/82#issuecomment-262892027, or mute the thread https://github.com/notifications/unsubscribe-auth/AAXS3Dk5XUnc5QJnZiYYoY_jfXCKrIHbks5rBoaAgaJpZM4KwZyI .
Yes, indeed very nice to get rid of registry use, which several others already wished for on the mailinglists!
Not sure if that can be within the scope of this QEP, but in an (our) organizational setting, we have some settings (e.g. connections to OWS) which are frequently used by almost all users. These were put into a default setting, which is the copied into the user-homer. Users then can adjust the settings and add more OWS and so on. However, OWS-url get updated now and then and the question arises how to update the URL in all user settings without overwriting changes/adjustments users made to their QGIS.ini.
What I would like to have would be the flexibility, that users can change the settings as they like, but that "central" settings can be updated by an administrator (and not every user has to do that himself). A kind of versioning of the QGIS.ini. Splitting up the Settings into different sections seems to be helpful in that direction.
But maybe that is something for a Python-Macro...
Hi, this is a very welcome change! +1 for standard location by default, + still a configpath option to move the profile elsewhere.
BTW, if we throw some light on profiles, I remember that some ressources can't be centralized easily by adding extra paths to qgis config. We currently can add additionnal paths to SVG folders, plugins folders, but we miss that for symbols (markers, color ramps aka palettes.), and processing models (and also themes, and project templates). Ressource sharing is great for individual users but not adapted for centralized systems. Currently, admins have to deploy complex ini file management strategies for those parts. Adding to extra paths options in QGIS would make life easier for many of us :). Sorry if I'm off topic.
Yes, I also like this idea's very much!!
Note that it is (at least on commandline in linux) already very easy using --configpath (which then takes BOTH settings and config). This also makes it easy to start with a clean slate or use different configs. If I remember correct @wonder-sk and @blazek did this for the configuration-work for schools.
I have QGIS config's for several client-projects within the client-project folder and start QGIS from command line: qgis --configpath /home/richard/zuidt/16/client_project/qgisfolder
In that way I also keep client-specific plugins/data/proxysettings etc etc together.
Firefox has the option to start a 'profile manager' using the -P option:
Off course for windows/mac it is probably better to have a second small binary for that?
Do you think we still would need 'application'-settings? Or a concept in which you can 'save' options either in 'current' config, or in the 'global'-config? I'm thinking about for example: the theme to use (I would never use the 'Vista'-theme anymore on windows for example), or the place where you put all your icons? We could have a concept in which QGIS first loads application settings and then overrides some of those via the config settings?
Another idea: I find the starting up of QGIS sometimes slow. So IF we put a 'config'-chooser in the startup process, some core part of QGIS can be started very quick (in parallel when the 'choose config' dialog is shown)... and then only starts what is nessecary. At least making the impression that we start up faster would be nice I think.
Another idea: IF we have this minimal config choose already, would it be an option (in case of a debug build) to stream the debug output to that window too? In that case it would make it easier for windows/mac users to see what is happening when something goes wrong. I always do debug builds, and when something goes wrong it is the first place I look into....
+1 from me, I cannot see any negative implications, and I see only clear advantages.
The --configpath
option is available on Windows too.
Hmm how did I not see the --configpath
option already. Good to know. I will update the QEP.
Most of this is still valid as I guess it's more about making this the default setup and I would still add a --profile
option which can just take a name of a profile to load.
--configpath
can be used for non profile based setups e.g save settings on a network drive or something.
@NathanW2 mmm, please do not overload the user with concepts: we already had 'config' and 'settings' and now we have 'profile' too?
@rduivenvoorde profile is a higher level concept a named config
if you will, most users will never load QGIS on the command line.
If it's better maybe we change all the naming from profile to config and use that? I'm not fixed on the profile naming so happy to roll with just calling it config.
@rduivenvoorde A config loader is a good idea, I will add that.
On the slow starting, I suspect this is due to the large start method in code and also in the plugins. A different topic that we will need to dig into.
@NathanW2 yeah, I'm ok with whatever naming, it's just that I do not want to overload people with (our) abstract ideas as what they want is just "have all my QGIS configurations" in one place. Do you also think there should be two levels of configurations (global and profile)?
Agreed, having different user profiles and the possibility of switching is a long standing idea, it would be great to see it implemented. Thanks.
Hi Nathan - A very welcome change!!
Just a couple of additions:
It would nice if it was possible to use some "variable substitution" mechanism inside the setup files.
Use Case: When I have to prepare a rollout of Qgis to a "enterprise size" number of users (250), I'll do the following:
In case the above is total gibberish, I have a detailed description of the process at https://github.com/Frederikssund/Alternativ-QGIS-installation
Suggestion no. 2: Put the microsoft support dll's ( msvcp.dll and msvcr.dll) in the qgis bin directory. That would make the Qgis program file directory "self contained" with no external file dependencies
Suggestion no 3: Add a user accesible "profile switcher", so it's possible to change profile dynamically during a session (A restart af Qgis it acceptable ;-)
Suggestion no 4: Loading the plugins sequentially (and not using threads) slows Qgis a lot during startup. A nice easy method to use threads during plugin initialization would help a lot !
Regards Bo Victor
Use Case: When I have to prepare a rollout of Qgis to a "enterprise size" number of users Pretty similar here compared to what @Frederikssund does and experiences.
A "profile switcher" would be great, because this is currently quite cumbersome on Windows esp. if one does not have admin rights...
For relative path / variable support there is also: https://hub.qgis.org/issues/12623
However, I am afraid discussion is drifting away from the original intention of the QEP...
Very good idea. Thanks for that.
The only negative implication that I can think of is multiplying the same plugin(s) in several different profiles. But not sure if that is really an issue.
@SrNetoChan I do not think the extra Mb wasted for multiple copies of the plugin are a problem. In fact, this would allow for testing new version, having different sets installed, etc.
The "profile switcher" idea is explored by Boundless: "http://boundlessgeo.github.io/qgis-plugins-documentation/profiles/plugin.html". However it would be nice if its baked into Qgis proper instead of being a plugin.
@haubourg yeah I think we will need to review and find any paths that are leaky and don't respect the config folder.
@SrNetoChan @pcav right it's not a big deal, you can set QGIS_PLUGINPATH
if you need global variables that are outside the users and install folder. This might come down to what @rduivenvoorde was saying about application-wide settings, although maybe we don't try and tackle this first and just limit the scope to a single config folder.
Any major objections to this QEP? Main stuff is having a single unified folder for all QGIS configs, moving away from the registry on Windows, etc.
No objection from me, seems a very good idea.
@NathanW2 @ninsbl no objections at all, in fact I'd like to propose a small enhancement (if you think that it's not in scope I can create a separate QEP). We are often in the need to produce custom builds that come with pre-configured settings. Currently, most of the URLs and other default settings values (like example WMS/WFS, official plugins repo etc.) are hardcoded inside C++ code, which is not optimal from the maintenance point of view and require a custom build just in case you need to change one of them. I'd like to see a way to provide a system-wide custom ini file that contains all default values for the settings.
... if we are keen to accept that the system-wide ini file for the defaults is mandatory, in other words that QGIS organization will provide an ini file with all the default values, we can avoid the inline fall-back values.
For a better handling of localized URLs (that should really not be in the translation files) we can also implement a simple string substitution system for the default values that would accept at least:
something like https://github.com/qgis/QGIS/pull/3911/commits/70da0fe1d8795c794fef105aaa5a108739828afb#diff-45491c675afd7cdbc0492b5859bcb03fR58
I'd be happy to work on this enhancements if you think it's in scope, if it's not, please tell me and I'll create a separate QEP.
@elpaso makes sense to me, +1
I do like that idea but let me see if it fits in this scope or is something extra.
On Tue, Jan 10, 2017 at 7:41 PM, Paolo Cavallini notifications@github.com wrote:
@elpaso https://github.com/elpaso makes sense to me, +1
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/qgis/QGIS-Enhancement-Proposals/issues/82#issuecomment-271529885, or mute the thread https://github.com/notifications/unsubscribe-auth/AAXS3FJZ2AbrBiceRuB0fuggZkZfMqedks5rQ1JRgaJpZM4KwZyI .
Started work on this here: https://github.com/NathanW2/QGIS/tree/qep_profiles
Current progress is loading profile from .default
file and loading with those settings. At the moment it's in %HOME% but I will move that location as per @nyalldawson comments.
@elpaso your comments about shard settings are still noted. I will rework the design to include those.
@NathanW2 , I was working on an independent draft but I'm more than happy to merge and contribute to yours.
Here are some implementation details and a working python prototype for a drop-in replacement of QSettings
. The subclass has some design limitations, but even if we go for a wrapper instead of a subclass, the internal implementation (which has tests) is still probably useful.
Introduce a new QgsSettings
class that subclasses the actual QSettings
but
can optionally provide default values taken from another ini file.
Values contained in the "default" ini file will provide the initial configuration and override possible inline defaults.
The override will support inheritance: a setting value in the default ini file will be overridden by a user setting.
The default ini file path will be searched in a standard location that will be overridable through an environment variable.
For example:
; defaults
[group]
key=value1
; user settings
[group]
key=value2
; will return key=value2
Groups will be inheritable, for example:
; defaults
[group]
key1=value1
key2=value2
; user settings
[group]
key3=value2
; a call to allKeys() will return [key1, key2, key3]
The subclass solution has the advantage of being a drop-in replacement of
QSettings
while maintaining the same API, but it has some disadvantages:
for example it would not be easy (without a singleton) to pass the path of
the default file from the QGIS application (i.e. implement a --defaultsinipath
command line option).
An alternate implementation would be adding a QgsSettings
instance to
QgsApplication
and use a new method (settings()
) to access the
QgsSettings
instance from QGIS and plugins.
"""
QSettings subclass that reads default values from an ini file
"""
import os
import unittest
from PyQt5.QtCore import QSettings
class QgsSettings(QSettings):
"""
A QSettings subclass that reads default values from an ini file
FIXME: this is a partial implementation: many methods are missing
"""
defaults = None
def __init__(self, orgname, appname, defaults_file=None):
super(QgsSettings, self).__init__(orgname, appname)
if self.defaults is None:
# Note: it will be SystemScope in the real implementation,
# here it is UserScope for testing
if defaults_file is not None:
if os.path.isfile(defaults_file):
self.defaults = QSettings(defaults_file, QSettings.IniFormat)
else:
self.defaults = None
else:
self.defaults = QSettings(QSettings.IniFormat, QSettings.UserScope, orgname, appname + '_defaults')
def beginGroup(self, group):
self.group = group
self.defaults.beginGroup(group)
super(QgsSettings, self).beginGroup(group)
def endGroup(self):
self.group = None
self.defaults.endGroup()
super(QgsSettings, self).endGroup()
def allKeys(self):
keys = set(super(QgsSettings, self).allKeys())
keys = keys.union(self.defaults.allKeys())
return keys
def childKeys(self):
keys = set(super(QgsSettings, self).childKeys())
keys = keys.union(self.defaults.childKeys())
return keys
def childGroups(self):
groups = set(super(QgsSettings, self).childGroups())
groups = groups.union(self.defaults.childGroups())
return groups
def value(self, key, default=None):
value = super(QgsSettings, self).value(key)
if value is None:
if self.defaults is not None:
value = self.defaults.value(key, default)
elif default is not None:
value = default
return value
class TestQgsSettings(unittest.TestCase):
cnt = 0
def setUp(self):
self.cnt += 1
self.settings = QgsSettings('itopen', 'itopen%s' % self.cnt)
def tearDown(self):
settings_file = self.settings.fileName()
settings_default_file = self.settings.defaults.fileName()
del(self.settings)
try:
os.unlink(settings_file)
except:
pass
try:
os.unlink(settings_default_file)
except:
pass
def addToDefaults(self, key, value):
defaults = QSettings(self.settings.defaults.fileName(), QSettings.IniFormat)
defaults.setValue(key, value)
defaults.sync()
def test_basic_functionality(self):
self.settings.setValue('itopen/name', 'elpaso')
self.settings.sync()
self.assertEqual(self.settings.value('itopen/name'), 'elpaso')
def test_defaults(self):
self.assertIsNone(self.settings.value('itopen/name'))
self.addToDefaults('itopen/name', 'elpaso')
self.assertEqual(self.settings.value('itopen/name'), 'elpaso')
def test_allkeys(self):
self.assertEqual(self.settings.allKeys(), set())
self.addToDefaults('itopen/name', 'elpaso')
self.settings.setValue('nepoti/eman', 'osaple')
self.assertEqual(2, len(self.settings.allKeys()))
self.assertIn('itopen/name', self.settings.allKeys())
self.assertIn('nepoti/eman', self.settings.allKeys())
self.assertEqual('elpaso', self.settings.value('itopen/name'))
self.assertEqual('osaple', self.settings.value('nepoti/eman'))
def test_precedence(self):
self.assertEqual(self.settings.allKeys(), set())
self.addToDefaults('itopen/names/name1', 'elpaso1')
self.settings.setValue('itopen/names/name1', 'elpaso-1')
self.assertEqual(self.settings.value('itopen/names/name1'), 'elpaso-1')
def test_groups(self):
self.assertEqual(self.settings.allKeys(), set())
self.addToDefaults('itopen/names/name1', 'elpaso1')
self.addToDefaults('itopen/names/name2', 'elpaso2')
self.addToDefaults('itopen/names/name3', 'elpaso3')
self.addToDefaults('itopen/name', 'elpaso')
self.settings.beginGroup('itopen')
self.assertEquals({'names'}, self.settings.childGroups())
self.settings.setValue('surnames/name1', 'elpaso-1')
self.assertEquals({'names', 'surnames'}, self.settings.childGroups())
self.settings.setValue('names/name1', 'elpaso-1')
self.assertEqual('elpaso-1', self.settings.value('names/name1'))
self.settings.endGroup()
self.settings.beginGroup('itopen/names')
self.settings.setValue('name4', 'elpaso-4')
keys = list(self.settings.childKeys())
keys.sort()
self.assertEqual(keys, ['name1', 'name2', 'name3', 'name4'])
self.settings.endGroup()
self.assertEqual('elpaso-1', self.settings.value('itopen/names/name1'))
self.assertEqual('elpaso-4', self.settings.value('itopen/names/name4'))
if __name__ == '__main__':
unittest.main()
The QGIS project will provide such a file with all the necessary entries to maintain the current behavior of the application but it will be possible for packagers to ship different default setting files and for sysadmins to provide a default configuration file.
/*
; defaults
[group]
key1=value1
key2=value2
[group2]
key4=value4
; user settings
[group]
key1=value-1
key3=value-3
*/
QgsSettings settings;
settings.value('group/key1', 'my value 1') == "value-1"; // true
settings.value('group/key3', 'my value 3') == "value-3"; // true
settings.value('group2/key4', 'my value 4') == "value4"; // true
settings.value('notexists/key8', 'my value 8') == "my value 8"; // true
@NathanW2 rethinking at the necessity to specify the "global-settings" (defaults) location, I'm not sure we really need it, I think we could also rely upon the fallback provided by Qt itself without the need to specify the path with a command line option (we could just set a different appname). It's supposed to be something read-only and outside of the user space anyway so providing a user configurable option does not make much sense in the first place.
Right. I think we just load that one from the install folder maybe.
On Mon, Jan 23, 2017 at 11:27 PM, Alessandro Pasotti < notifications@github.com> wrote:
@NathanW2 https://github.com/NathanW2 rethinking at the necessity to specify the "global-settings" (defaults) location, I'm not sure we really need it, I think we could also rely upon the fallback provided by Qt itself without the need to specify the path with a command line option (we could just set a different appname). It's supposed to be something read-only and outside of the user space anyway so providing a user configurable option does not make much sense in the first place.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/qgis/QGIS-Enhancement-Proposals/issues/82#issuecomment-274487336, or mute the thread https://github.com/notifications/unsubscribe-auth/AAXS3EPVa0YYEAwJliKIt4_jP5ky5HG7ks5rVKqqgaJpZM4KwZyI .
Hi @NathanW2 and @elpaso,
@NathanW2 wrote:
Right. I think we just load that one from the install folder maybe.
Seems like a logical place for the default ini file is in QgsApplication::pkgDataPath()
? Generally, this would require admin user permissions to change after install (relative to install location and type, of course), which I think is appropriate for such a file.
Followup to my last post. The idea behind having the default ini part of the QGIS install is that such a file should IMO be relative to the install package. If it is in a standard Qt-known location, then it will be shared across QGIS installs, which is sometimes nice to have, but often causes more problems than it solves, especially if it gets stomped by a subsequent (or even an older) install. It is not uncommon for users to have multiple QGIS installs and isolation should be considered here.
@dakcarto right so we agree it should go into the install folder, so you don't share between installs as that would be a pain to manage.
I will update this QEP tonight with more notes on the final design
Here is a first C++ quick and dirty prototype bound to a single setting as a proof of concept. https://github.com/boundlessgeo/QGIS/commit/042abdf44646ba0605b6ab4253f7a098e48c4809 Note: I developed this as a proof of concept before our yesterday's conversation, I'm posting the link just in case we can re-use something.
@elpaso that looks good to me. mind if I just yank that into my code?
@NathanW2 not at all, I can port the python tests to C++ if you like, or write the bindings and use Python for the tests too. Currently missing from the implementation are the array read/write methods though.
@dakcarto @elpaso how do you feel about the machine settings using the same format as what user profiles do? e.g on windows it might look like this:
This makes it easy to "deploy" machine settings by just building it up as a local config and then putting it in the install folder?
Does that make sense?
We might have do some more advanced merging logic for some of the other things but I think it should work ok.
@rduivenvoorde I have update the QEP to now call it "configs" and not profiles to avoid any confusion with computer profiles and logins.
@NathanW2 looks good to me. We need an easy way to create the default (machine) settings and I cannot think of an easiest one than copying from and existing configuration.
@NathanW2 wrote:
how do you feel about the machine settings using the same format as what user profiles do?
That seems fine to me, especially if inheritance (in @elpaso's prototype) is to work properly.
Maybe there could be checkboxes added to the Options -> Advanced -> Advanced Settings Editor
, then one could just select what to export to a custom settings file in INI format (i.e. the QSettings-style INI that is, plus anything needed for custom configs in this QEP). Might be a simple way to generate the default (install-relative) settings file.
Or, do you mean the install-relative defaults config would actually be a directory, instead of a single config file?
Yeah it would just be a folder with the same layout as the users config but in the install folder.
On Tue, Jan 31, 2017 at 9:32 AM, Larry Shaffer notifications@github.com wrote:
Or, do you mean the install-relative defaults config would actually be a directory, instead of a file?
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/qgis/QGIS-Enhancement-Proposals/issues/82#issuecomment-276226458, or mute the thread https://github.com/notifications/unsubscribe-auth/AAXS3M0DmIVmJP-R6HQwFnXYkrZgGZz4ks5rXnMcgaJpZM4KwZyI .
QGIS Enhancement 82: Unified Application Configuration
Date 2016/11/12 Author Nathan Woodrow @NathanW2 Contact woodrow.nathan@gmail.com maintainer @NathanW2 Version QGIS 3.0
Summary
This QEP is to add and expand Unified Application Configuration support into core QGIS. The main idea is to bundle everything that is normally in
.qgis2
and settings (registry on windows, config files on OS X/*nix, etc) into a single folder in the users platform based settings folder. This would allow the creation of saved config folder, you can think of them like profiles in Chrome/Firebreak, etcThe config folder would be a single folder with all
.qgis2
and settings files which currently the result of doing the following in QGIS 2:qgis --configpath folder\qgis
As saved configs contain isolated settings and plugins they can be great for different workflows, demos, users of the same machine, or testing settings, etc.
Proposal includes removing the use of current
.qgis2
and settings paths (registry on Windows) in QGIS 3.x in favour of new profiles folder setup.I think this method of storing settings and plugins, etc, could remove a lot of current confusion in how QGIS stores settings, as well as increase the flexibility in enterprise style deployments of QGIS.
Backups of user profiles also because very easy. A simple copy the profile folder to make a backup.
Proposed Solution
This proposal is to add new option
--profile
, as well as UI related to creating and switching saved configs.All configs would live in a single top level folder in the platform dependent location e.g appdata on windows, etc, A link the about screen would also have the option to locate the current profile folder in the OS file manager.
An example of profile layout:
By default, QGIS would run using the default config which is found in
.default
, e.gprofile1
Part of this QEP is also loading machine-wide settings that are installed in the QGIS install folder. These should be loaded and then joined with the users saved config settings. Settings will follow the same format as the a users folder setup however there is only one.
Affected Files.
main()
of QGIS would be altered to load configs from the settings folder.A new class will also be introduced called
QgsSettings
which hides the implementation of config loading away from the code base.QgsSettings
will be used in place ofQSettings
though out the core code base.QgsSettings
will have an API similar toQSettings
in order to be a drop in replacement. The current implementation is to just be a wrapper forQSettings
however in the future this could hold other logic for finding settings.Performance Implications
None.
Further Considerations/Improvements
Cloud config
By hiding any implementation details inside
QgsSettings
we can also add loading config from a database and webserver in the future.Webserver could expose endpoints to download/upload zip files of the user's config folder into a hosted environment. This is out of the scope of this QEP, however, this can be used as a building block for that function in future.
Backups
Built in backup process to allow rollbacks of settings. As the settings file is simply text files each change and is backed up and compressed for later rollback.
Backwards Compatibility
If QGIS 3.0 will inherit the settings of QGIS 2.x would migrate any existing settings and plugins into the new profile folder layout.
Issue Tracking ID(s)
(optional)