Open akbargumbira opened 8 years ago
Nice first draft, thanks @akbargumbira . @rduivenvoorde here is the info you were after regarding what is being planned for the style repo. Please add your thoughts if you have any.
Thanks @akbargumbira
Here are few random thoughts:
XML-RPC
lib we are using on the plugins website The specs for the client GUI side are still missing, we should try to sketch them out ASAP (to the top of my mind @anitagraser , @NathanW2, @nyalldawson and @rduivenvoorde could probably give more hints on this) Here is a first incomplete list of functions I'd like to see in the plugin:
I believe that you can copy/paste the models, permissions config and authentication stuff from the plugins repo, we are using osgeo id on the plugins website, I'd like to see that kind of authentication here too.
Please note that this repo will (hopefully) have a heavy traffic, caching should be activated wherever it makes sense. Avoid query string params and use path parameters when possible.
I'd set a target for current master or 2.16 but if we are going to implement the client as a python plugin there shouldn't be any problem in supporting 2.14 too.
In the plugin website, we've been using a validator class to check for minimum requirements on the submitted plugins. I'd suggest to do the same here and to make this an independent package, so that we can include it in the client plugin. This will make much more easier for a style/symbol submitter to check if their work is suitable for upload on our repo before they actually submit them.
My thoughts:
@pcav yes, a list of available tags is a must have Can you explain what you mean with "mass import entire sets...": from what kind of sources? And yes, a validator routine with sanity checks is definitely required.
@elpaso there are nice symbol sets around with free licence (I have opened a ticket listing some of the potentially useful ones, I can search it back when useful). It would be good to have the possibility of importing the full set, maybe as a zipfile.
I need to sit down and read this fully yet but just on this note:
QGIS Desktop client (python plugin) No plugins please Python or C++. What ever we come up with needs to be baked into QGIS as a fire class core feature. The style manager could be extended to provide download ability or we build a whole new tool for it, but no plugins.
On Thu, Apr 28, 2016 at 8:30 PM, Paolo Cavallini notifications@github.com wrote:
@elpaso https://github.com/elpaso there are nice symbol sets around with free licence (I have opened a ticket listing some of the potentially useful ones, I can search it back when useful). It would be good to have the possibility of importing the full set, maybe as a zipfile.
— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/qgis/QGIS-Enhancement-Proposals/issues/58#issuecomment-215380591
first class**
On Thu, Apr 28, 2016 at 8:35 PM, Nathan Woodrow madmanwoo@gmail.com wrote:
I need to sit down and read this fully yet but just on this note:
QGIS Desktop client (python plugin) No plugins please Python or C++. What ever we come up with needs to be baked into QGIS as a fire class core feature. The style manager could be extended to provide download ability or we build a whole new tool for it, but no plugins.
On Thu, Apr 28, 2016 at 8:30 PM, Paolo Cavallini <notifications@github.com
wrote:
@elpaso https://github.com/elpaso there are nice symbol sets around with free licence (I have opened a ticket listing some of the potentially useful ones, I can search it back when useful). It would be good to have the possibility of importing the full set, maybe as a zipfile.
— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/qgis/QGIS-Enhancement-Proposals/issues/58#issuecomment-215380591
Maybe better not to limit this proposal to styles and symbols, but also leave room for other resources like scripts, models, composer templates, etc?
Yeah, maybe something like a generic "Online Resources" thing.
On Thu, Apr 28, 2016 at 8:58 PM, Alexander Bruy notifications@github.com wrote:
Maybe better not to limit this proposal to styles and symbols, but also leave room for other resources like scripts, models, composer templates, etc?
— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/qgis/QGIS-Enhancement-Proposals/issues/58#issuecomment-215388763
@NathanW2 what's wrong with a Python plugin implementation? I was thinking at something like the plugin manager, a core plugin written in Python. But if @akbargumbira thinks has the time and skills to do that in C++ I'm not against it.
@alexbruy and all: please note that this is GSOC project, with limited time and resources, IMHO we have better chances to have a useful outcome if we keep the scope limited.
Ok if it's a skill/time thing. I will review this QEP and get back back with any notes.
What do you think should be the main focus on this GSOC. Is there more benefit on the server side and leaving the client. The client should be easy-ish provided the API from the server is easy to use.
I'm pretty confident that @akbargumbira , with the help and support of the QGIS community will have enough time to build production-ready symbols/styles server and client and to test-deploy it. The server side is mostly defined, the client is not, we have time until the 23 may to define the specs for the client and your help and suggestions are highly appreciated. I feel that implementing the client side in Python would be much more easier and quicker, that's why I'm pushing in that direction. For the server services, If the time is not enough, we could always add more items (scripts etc., like suggested by @alexbruy) later.
@elpaso my comment was not about implementing support for all kinds of resources during GSoC, but mainly about developing extensible platform, so support of other resources can be added easily later.
@elpaso:
categories should be a hierarchy an item (style, symbol etc.) belongs to one or more categories
Noted, I'll probably just use django-mptt (https://github.com/django-mptt/django-mptt) for that
item name should probably have a unique constraint, or at least, must be unique in its categories (for multiple symbols XML we should automatically assign incremental number suffixes to the item name or use a similar system)
I am not really sure about this that the name should be unique. i think resources are likely to have similar/the same name, e.g for SVG, we could expect that there will be more than one SVG for hospital, right?
do we really want to support private items?
Now that I think about the purpose of this repository, I think it's not necessary :) I'll remove it from the model
The specs for the client GUI side are still missing, we should try to sketch them out ASAP
Yes, I'll make the mock-up this weekend following the feedback given from this QEP
I believe that you can copy/paste the models, permissions config and authentication stuff from the plugins repo, we are using osgeo id on the plugins website, I'd like to see that kind of authentication here too.
I am going to stick to the model above instead of using the scheme from plugins repo. I would like to have is_trusted as a field instead of permission (For the resources, I don't want to have 3 different permissions for each type of resources. When someone gets trusted, it means (s)he could upload it directly all type of resources. I don't like global permission either, hence I added that field). But apart from that, I think more or less I could copy from plugin project for authentication. I'll start to put some layout code including this, this weekend.
Please note that this repo will (hopefully) have a heavy traffic, caching should be activated wherever it makes sense. Avoid query string params and use path parameters when possible.
Ok. Noted
In the plugin website, we've been using a validator class to check for minimum requirements on the submitted plugins. I'd suggest to do the same here and to make this an independent package, so that we can include it in the client plugin. This will make much more easier for a style/symbol submitter to check if their work is suitable for upload on our repo before they actually submit them.
Do you suggest to add a field like 'minimum_qgis_version' for the Resource model above and user needs to specify this when uploading the resources? For style, we can extract 'qgis_version' from the qml file though it only represents the QGIS version used when importing the style
@pcav
My thoughts:
free tags have the potential of creating lots of noise; perhaps better to first suggest a lit of existing or default ones, not encouraging too much the addition of new ones
One resource has categories (which will be chosen from tree structure) and free tags. Do you suggest to remove free tags from the model?
there should be a method to mass import entire sets from other sources
Noted. But I won't make this as a high priority for GSoC duration. I'm not going anywhere and probably could continue the work after GSoC on my free time.
a licence and author should be made explicit
For license, do you suggest to add it for SVG or also for style (.qml) and symbol definition (.xml) as well (these both are imported from QGIS)?
is there a chance of adding malicious code to svg/xml? If so, a filter/check should be added. Thanks!
Noted. I'll explore more about this first.
@NathanW2 what's wrong with a Python plugin implementation? I was thinking at something like the plugin manager, a core plugin written in Python. But if @akbargumbira thinks has the time and skills to do that in C++ I'm not against it.
Personally I prefer more developing with Python (I am not fluent in C++ and it's been a long time I worked with it).
@alexbruy yes, I'll keep in mind that this platform should be able to be extended with other kind of resources.
No plugins please Python or C++. What ever we come up with needs to be baked into QGIS as a fire class core feature.
I'm with Nathan here - I'm ok if this is done in python, but please DON'T make it a plugin. It should be part of core just like the python console.
free tags have the potential of creating lots of noise; perhaps better to first suggest a lit of existing or default ones, not encouraging too much the addition of new ones
One resource has categories (which will be chosen from tree structure) and free tags. Do you suggest to remove free tags from the model?
I think the way stackexchange does it is great. It's free form but proposes tags and offers additional descriptions for tags (tooltips). I would rather remove categories.
Very nice to read this draft, it will be great to see this improvement.
I'm with Nathan here - I'm ok if this is done in python, but please DON'T make it a plugin. It should be part of core just like the python console.
I agree, this should definitely be core-functionality which is not opt-in like plugins.
If this is mostly implemented in python, it would be nice if the gui libraries don't depend on python code and the setup code is run in app. (I.e. it should be done the console way and not the edit forms way)
It would be great if Python 3 and Qt 5 could be taken into consideration right from the beginning. There are a couple of technologies which are deprecated and could potentially be used (e.g. don't use QtScript) just to be sure we don't implement things now on technology that is soon going to disappear.
On the other hand others require new Qt versions that we don't yet support but that could be interesting (e.g. http://doc.qt.io/qt-5/json.html). If there is use for it, I think it can be discussed to do so. Basically depends on how you feel it @akbargumbira =)
Looking forward to downloading a lot of beautiful symbols from there!
@m-kuhn I also think that this must be in the core and not an optional feature. But I'd prefer a python implementation. What I had in mind was a core python plugin, just like "plugin manager", but if there are any other ways to do a python implementation that is not a (core) plugin it's fine for me.
@akbargumbira for the permissions: you can have 3 (or more) different permissions and set/unset them as a whole, but it's definitely up to you, just note that if you choose the user model way you'll need a custom user model, while if you stick with the Django permission model you can just use its API.
For the validator, let me try to explain what I mean: we do need validation for the uploaded items, there will probably be a set of rules that the uploaded asset should pass before we decided if it is valid and accept it in the repo. If you design this validation routine as an independent module, you can re-use it in the client (the QGIS side) to perform the checks before the user tries to upload the asset to the repo. This would be very useful for the users that will know in advance if their symbol/style will be accepted in the repo and they will able to fix any error/problem in advance.
I think it was just a mix up of terms. I'm fine with a Python version e.g plugin manager, console styles, they are just features that are called from core code. Unlike a plugin that is handled generic for all and can be turned on and off.
So +1 to do it in Python for it.
On Fri, 29 Apr 2016 10:38 pm Alessandro Pasotti notifications@github.com wrote:
@m-kuhn https://github.com/m-kuhn I also think that this must be in the core and not an optional feature. But I'd prefer a python implementation. What I had in mind was a core python plugin, just like "plugin manager", but if there are any other ways to do a python implementation that is not a (core) plugin it's fine for me.
@akbargumbira https://github.com/akbargumbira for the permissions: you can have 3 (or more) different permissions and set/unset them as a whole, but it's definitely up to you, just note that if you choose the user model way you'll need a custom user model, while if you stick with the Django permission model you can just use its API.
For the validator, let me try to explain what I mean: we do need validation for the uploaded items, there will probably be a set of rules that the uploaded asset should pass before we decided if it is valid and accept it in the repo. If you design this validation routine as an independent module, you can re-use it in the client (the QGIS side) to perform the checks before the user tries to upload the asset to the repo. This would be very useful for the users that will know in advance if their symbol/style will be accepted in the repo and they will able to fix any error/problem in advance.
— You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub https://github.com/qgis/QGIS-Enhancement-Proposals/issues/58#issuecomment-215700014
@NathanW2 fully agreed, sorry for being not clear with the terms.
Some wild brainstorming here
I'm probably (too) late, but I'm still not sure which main goal we are actually have here:
What about keeping the client (QGIS) side then as simple as possible (so no actual extra gui stuff there?), for example: for QGIS you just download a zip with icons, unzip them in a dir with the zipname, and put that dir in your svg-icon path. Nothing more: no reading of metadata. For QGIS the icons just stay 'dumb' image files (like the icons we currently have).
In that way we can cut up complexity for now, leaving (search, metadata, svg-checking...) complexity as much as possible on the server. In that way we also keep room for very (personal) simple icon-repo's (like github??) which for example only offer a couple of thematic zip's.
There are a lot of use-cases with: icon-theme's, searchable icon libraries, dynamic icon building repo's (in which you can choose certain styling/coloring scheme's) etc. We should not try to bring all this knowledge into QGIS in my opinion... Will the average user for example care about the license for an individual icon? In my experience he/she is just looking/searching for a zip with a coherent set of icons usable for his/her current GIS-work
/end brainstorm, feel free to ignore
In my opinion, the main goal of this work is to create an easy way for users to share and access symbols (SVGs) and styles (QML). Currently, it is only possible to load a symbol XML from URL/file path using Style Manager:
The issue here (besides lack of SVG and QML support) is discoverability. Users have to know that a specific symbol XML exists and where to get it. A new tool should therefore make it easy to discover available resources, e.g. suggest most popular resources and support search by name, tags, author, ...
Imho, the GUI should definitely display previews.
(Sidenote: There is already a solution to share Processing Python scripts in the Processing toolbox.)
One other thing to think about is the possibility of project portability. I'm thinking of the use case here where a user creates a project with a bunch of downloaded symbols and then uploads it to the server. It would be very nice if any published symbols were just automatically fetched if they are not present on the file system when the project is loaded. We could have a GUID for each symbol so that it could be retrieved canonically.
Amongst @rduivenvoorde's brain storming ideas, I find it quite intriguing the idea of having a distributed repo like system so that users can host their own public symbology repos. Though I guess the complexity goes up a whole heap if we do something like that so the work may not be worth the reward...unless some how @akbargumbira can come up way to make such a thing really simple and clear.
feature implemented so maybe, should we close this QEP
QGIS Enhancement 58: QGIS Styles, Symbols, and SVG Markers Sharing Repository
Date 2016/04/27
Author Akbar Gumbira (@akbargumbira)
Contact akbargumbira@gmail.com
Status Draft
Version QGIS X.X
Summary
In this GSoC project, I will focus on making styles (.qml), symbols definition (.xml), and SVG markers sharing possible through web services that later can be accessed and used in QGIS. The project scope includes:
This project was partly done (styles and symbols sharing) in GSOC 2013, but unfortunately, the work was not merged in to master as some QGIS core developers thought that it was not production-ready, mainly on the server side. For the server side, I have moved the previous GSOC 2013 work to this repository: http://github.com/akbargumbira/QGIS-Sharing and will continue to work there.
GSoC 2013 Work
This is the work done on GSoC 2013. For symbol, user uploads xml file itself and the list of tags. For the styles, user submits a form with name, QML file, description, and tags.
Proposed Solution
Server Side
Authentication and user permissions
The behavior for authentication will follow what we have in Django plugin. User logs in using OSGeo acount.
User Management
Resource Management
Resource Classification
Model
This section explains about the proposed model on the server side.
Author
Notes:
Category
Category will be hierarchical using django-mptt module (https://github.com/django-mptt/django-mptt)
Notes:
Resource
This is an abstract model.
Symbol
This model inherits Resource abstract model.
Notes:
Styles
This model inherits Resource abstract model.
Notes:
SVG Markers
This model inherits Resource abstract model.
Notes:
Client Side - QGIS Desktop client (python plugin)
In QGIS Desktop, user should be able to:
Notes from Mailing List
Further Considerations
Backwards Compatibility
User will need to upgrade to the QGIS target version to be able to use this new feature.
Votes
(required)