Rename Groups to Themes

[riccardoerra]

Other CKAN websites use the concept of Themes instead of Groups. How to achieve that?

Support received

Renaming groups to themes

To rename groups to themes, you need to remap the URL structure so that /group/ redirects to /theme/ and then modify the templates that include "group" or "groups" in their output. In most cases you'll basically just be making a copy of the relevant group template, changing the labels from group to theme, and that's it. You'll probably want to change the labeling for the search facets as well.

In some cases, the system will still automatically generate a /group link in the front end (for a button, for instance). Finding and changing all the auto-generation code is bit of a pain and, if your URL remapping is done correctly, it won't matter. For example, a button will link to /group, but if a user clicks on it, they'll wind up at /theme, and it will only matter if care what the tooltip or href value is in the event that someone inspects the element.

There are a couple of things that people usually don't change, like the API: standard practice is not to touch internals like the API and search, so you'd still use group_list to get a list of the "themes" and when faceting a search on themes, the URL in the address bar will still be {site_url}/dataset?q={search_term}&group={ecology}. You could add new API endpoints and tweak the search, but that's a lot of additional work that may not be necessary (and so far, in our experience, it hasn't been).

For the remapping: In your plugin.py module you'll need to import routes.mapper. You'll also need to add plugins.implements(plugins.IRoutes) to your Iod_ThemePlugin. Then you'll be able to add a method, before_map(self, map), which remaps the URLs. Below you'll find an example of how that method might look:

def before_map(self, map):
    map.redirect('/group', '/topic',
                 _redirect_code='301 Moved Permanently')
    map.redirect('/group/{url:.*}', '/topic/{url}',
                 _redirect_code='301 Moved Permanently')
    group_controller = 'ckan.controllers.group:GroupController'
    with routes.mapper.SubMapper(map, controller=group_controller) as m:
        m.connect('topic_index', '/topic', action='index')
        m.connect('/topic/list', action='list')
        m.connect('/topic/new', action='new')
        m.connect('/topic/{action}/{id}',
                  requirements=dict(action='|'.join([
                      'delete',
                      'admins',
                      'member_new',
                      'member_delete',
                      'history'
                      'followers',
                      'follow',
                      'unfollow',
                  ])))
        m.connect('topic_activity', '/topic/activity/{id}',
                  action='activity', ckan_icon='time')
        m.connect('topic_read', '/topic/{id}', action='read')
        m.connect('topic_about', '/topic/about/{id}',
                  action='about', ckan_icon='info-sign')
        m.connect('topic_read', '/topic/{id}', action='read',
                  ckan_icon='sitemap')
        m.connect('topic_edit', '/topic/edit/{id}',
                  action='edit', ckan_icon='edit')
        m.connect('topic_members', '/topic/edit_members/{id}',
                  action='members', ckan_icon='group')
        m.connect('topic_bulk_process',
                  '/topic/bulk_process/{id}',
                  action='bulk_process', ckan_icon='sitemap')
    return map

Remember that if remapping a particular URL that CKAN gives you trouble, you can also change the URLs that CKAN generates directly in the templates themselves.

To change groups to themes on the front end: You'll also need to change the group templates so that they refer to themes rather than groups. You already know how to do that (and the theming guide has more to say on the topic (http://docs.ckan.org/en/latest/theming/index.html)). Most of the templates you'll need to extend or override are in /templates/groups. Since under the hood you're still using groups and just remapping/renaming things, the modified templates will still have the same names and folder structure, i.e., the folders and file names will still refer to groups. You'll also probably want to change /templates/snippets/search_result_text.html and /templates/header.html. You can search for other templates you want to modify. You may also want to adapt some of the URL generating functions like {{ h.build_nav_icon('topicactivity', ('Activity Stream'), id=c.group_dict.name, offset=0) }} Note that in this case, you'd change the first argument, but not the third, since CKAN is still working with the group controller and group variables under the hood.

One thing to watch out for: the GroupController has a method, _guess_group_type, which tries to guess the type of group based on the URL (it's possible to have more than one type of group). When you rename groups to themes, this method will return theme or themes, depending on your URL schema, rather than group, which is what the rest of the controller will be expecting. The easiest workaround is just to override the GroupController with a copy in your plugin (or patch it directly in CKAN itself) that method only returns 'group'. There are other ways you could tackle the issue, but that's probably the fastest.

Regarding renaming groups to themes, you just need to add import routes.mapper to the imports. Routes is one of the dependencies in the requirements.in file. If you're getting errors, it might not be installed, or the errors might be occurring because you did import routes.mapper as mapper and didn't change line 7 of the code snippet we sent, which refers to routes.mapper.Submapper. If Routes is installed and you're still getting an error when you try to do import routes.mapper, please send the error message so we can try to debug.

[riccardoerra]

Still not working.

Routes is installed:

pip show Routes
Name: Routes
Version: 1.13
Summary: Routing Recognition and Generation Tools
Home-page: http://routes.groovie.org/
Author: Ben Bangert
Author-email: ben@groovie.org
License: UNKNOWN
Location: /home/vagrant/lib/python2.7/site-packages
Requires: repoze.lru

Imported routes.mapper like this: https://github.com/smallmedia/iod-ckan/blob/d887292b63a5ffeb20e634cc8aa401ff00dc3dcc/ckanext-iod_theme/ckanext/iod_theme/plugin.py#L6

Changed routes.mapper.SubMapper to mapper (tried also other options): https://github.com/smallmedia/iod-ckan/blob/d887292b63a5ffeb20e634cc8aa401ff00dc3dcc/ckanext-iod_theme/ckanext/iod_theme/plugin.py#L77

But still getting error when I run paster serve /etc/ckan/default/ckan.ini:

Traceback (most recent call last):
  File "/home/vagrant/bin/paster", line 11, in <module>
    sys.exit(run())
  File "/home/vagrant/local/lib/python2.7/site-packages/paste/script/command.py", line 102, in run
    invoke(command, command_name, options, args[1:])
  File "/home/vagrant/local/lib/python2.7/site-packages/paste/script/command.py", line 141, in invoke
    exit_code = runner.run(args)
  File "/home/vagrant/local/lib/python2.7/site-packages/paste/script/command.py", line 236, in run
    result = self.command()
  File "/home/vagrant/local/lib/python2.7/site-packages/paste/script/serve.py", line 284, in command
    relative_to=base, global_conf=vars)
  File "/home/vagrant/local/lib/python2.7/site-packages/paste/script/serve.py", line 329, in loadapp
    **kw)
  File "/home/vagrant/local/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 247, in loadapp
    return loadobj(APP, uri, name=name, **kw)
  File "/home/vagrant/local/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 272, in loadobj
    return context.create()
  File "/home/vagrant/local/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 710, in create
    return self.object_type.invoke(self)
  File "/home/vagrant/local/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 146, in invoke
    return fix_call(context.object, context.global_conf, **context.local_conf)
  File "/home/vagrant/local/lib/python2.7/site-packages/paste/deploy/util.py", line 58, in fix_call
    reraise(*exc_info)
  File "/home/vagrant/local/lib/python2.7/site-packages/paste/deploy/compat.py", line 23, in reraise
    exec('raise t, e, tb', dict(t=t, e=e, tb=tb))
  File "/home/vagrant/local/lib/python2.7/site-packages/paste/deploy/util.py", line 55, in fix_call
    val = callable(*args, **kw)
  File "/vagrant/ckan/config/middleware/__init__.py", line 45, in make_app
    load_environment(conf, app_conf)
  File "/vagrant/ckan/config/environment.py", line 97, in load_environment
    p.load_all()
  File "/vagrant/ckan/plugins/core.py", line 139, in load_all
    load(*plugins)
  File "/vagrant/ckan/plugins/core.py", line 167, in load
    plugins_update()
  File "/vagrant/ckan/plugins/core.py", line 121, in plugins_update
    environment.update_config()
  File "/vagrant/ckan/config/environment.py", line 198, in update_config
    routes_map = routing.make_map()
  File "/vagrant/ckan/config/routing.py", line 106, in make_map
    map = plugin.before_map(map)
  File "/vagrant/ckanext-iod_theme/ckanext/iod_theme/plugin.py", line 77, in before_map
    with mapper(map, controller=group_controller) as m:
TypeError: 'module' object is not callable

[kmbn]

You had with mapper(map, controller=group_controller) as m: rather than with mapper.SubMapper(map, controller=group_controller) as m: (which works if you import routes.mapper as mapper) or with routes.mapper.SubMapper(map, controller=group_controller) as m: as in the example. I also added the after_map method, which is missing from the example.

[riccardoerra]

Things still pending

• How to change ckan_icon='group'? Tried in plugin.py but no effect.

Depending on the case, it's either in plugin.py or in the template file e.g. {% link_for _('View showcase'), controller='ckanext.showcase.controller:ShowcaseController', action='read', id=pkg.name, class_='btn', icon='eye-open' %} {{ h.build_nav_icon('ckanext_showcase_edit', _('Edit showcase'), id=pkg.name) }}

• controller='group' or controller='theme', type='group' or type='theme'? For now I left everything as group because changing to theme triggered errors.

• group_count/group_list??? Wanting to have the stats block in the homepage to output the number of theme created.

Depending on how they're created, either stats.group_count or stats.organization_count

• Can create theme but can't view theme created http://iod-ckan-live.aws.smallmedia.org.uk/theme/theme error gives: 404 Not Found. The resource could not be found. Incorrect group type.

http://iod-ckan-live.aws.smallmedia.org.uk Does not open for me, but an 'Incorrect group type' is related to another question "Override the GroupController with a copy in your plugin that method only returns 'group'"

• Override the GroupController with a copy in your plugin that method only returns 'group'??? Not sure I understand what to do here.

Copy the GroupController from CKAN core in your code, and make the neccessary changes to only return group type: group (if that fits your use-case with how you use/implement groups/organizations.)

• Activity stream strings are in .po files? Noticed that the copy for some sections of CKAN are in the trans file. I guess I need to change from group to theme in there.

Yes, some of the strings are marked for translation, and therefore appearances of 'Group' also need to be changed in translation files as well. (Or change them directly in the templates, recompile translation files and make changes if they are necessary )

• group_create or similar not present in plugin.py Everything group_xxx needs to be added in plugin.py and then changed to theme_xxx in the templates?

Yes, everything changed in plugin.py (for example group_show to theme_show) needs to be changed in the templates as well. For example instead of: {{ h.build_nav_icon('group_edit', _('Edit'), id=c.group_dict.name) }} {{ h.build_nav_icon('theme_edit', _('Edit'), id=c.group_dict.name) }}

• How to change URL slug?

(I’m assuming you don't want /group to appear in the slug preview)

Please view the following:

templates/package/snippets/package_basic_fields.html templates/group/snippets/group_form.html templates/organization/snippets/organization_form.html

There are many ways to handle this, the simplest would probably be http://jinja.pocoo.org/docs/2.9/templates/#replace

• How to change Groups in search sidebar? Can't find related template.

templates/snippets/facet_list.html Quickest solution would again be http://jinja.pocoo.org/docs/2.9/templates/#replace, if you want to avoid digging up the backend.

• Is it possible to purge deleted groups/themes?

Yes, please view: http://docs.ckan.org/en/ckan-2.6.3/api/index.html#ckan.logic.action.delete.group_purge

[riccardoerra]

Things still pending [09/10/2017]

1. How to change ckan_icon='group'?

Tried in plugin.py but no effect.

Depending on the case, it's either in plugin.py or in the template file e.g. {% link_for _('View showcase'), controller='ckanext.showcase.controller:ShowcaseController', action='read', id=pkg.name, class_='btn', icon='eye-open' %} {{ h.build_nav_icon('ckanext_showcase_edit', _('Edit showcase'), id=pkg.name) }}

Managed to change icon almost everywhere but can't find a way to change it for activity stream.

The icon for activity stream is defined in routing.py (in CKAN) or plugin.py (in an extension). Whichever icon is defined in plugin.py - for example: m.connect('group_activity', '/group/activity/{id}/{offset}', action='activity', ckan_icon='time'), in the template, the build_nav_icon helper will use the icon defined in plugin.py, (for example group/read_base.html)

will use the 'time' icon defined in plugin.py

2. controller='group' or controller='theme', type='group' or type='theme'?

For now I left everything as group because changing to theme triggered errors. Do I need to change every controller and type from group to theme? Would be nice to know/understand which ones need to be changed and which ones don't.

To replace 'group' with 'theme' :

Replace group actions in plugin.py with 'theme' map.redirect('/group', '/theme', _redirect_code='301 Moved Permanently') map.redirect('/group/{url:.*}', '/theme/{url}', _redirect_code='301 Moved Permanently')

group_controller = 'ckanext.yourextension.path.to.controller:ControllerName'

with SubMapper(map, controller=group_controller) as m: m.connect('theme_index', '/theme', action='index') ...etc. in controller.py (or wherever you write your controller classes) import ckan.controllers.group as group

class ThemeController(group.GroupController): group_types = ['group']

def _guess_group_type(self, expecting_name=False): return 'group'

3. group_count/group_list???

Wanting to have the stats block in the homepage to output the number of theme created.

Depending on how they're created, either stats.group_count or stats.organization_count

What do I have to change here? Having controller='theme' doesn't do anything.

<a href="{{ h.url_for(controller='group', action='index') }}">
    <b>{{ h.SI_number_span(stats.group_count) }}</b>
     {{ _('theme') if stats.group_count == 1 else _('themes') }}
</a>

I think implementing the previous point should fix this. stats.group_count outputs the length of the output from the group_list API call.

4. Override the GroupController with a copy in your plugin that method only returns 'group'???

Not sure I understand what to do here.

Copy the GroupController from CKAN core in your code, and make the neccessary changes to only return group type: group (if that fits your use-case with how you use/implement groups/organizations.)

Found the GroupController in ckan/controller/group.py but not sure what/where to charge. Something to do with _guess_group_type I guess?

To replace 'group' with 'theme' :

Replace group actions in plugin.py with 'theme' map.redirect('/group', '/theme', _redirect_code='301 Moved Permanently') map.redirect('/group/{url:.*}', '/theme/{url}', _redirect_code='301 Moved Permanently')

group_controller = 'ckanext.yourextension.path.to.controller:ControllerName'

with SubMapper(map, controller=group_controller) as m: m.connect('theme_index', '/theme', action='index') ...etc. in controller.py (or wherever you write your controller classes) import ckan.controllers.group as group

class ThemeController(group.GroupController): group_types = ['group']

def _guess_group_type(self, expecting_name=False): return 'group'

5. How to change URL slug?

(I’m assuming you don't want /group to appear in the slug preview)

Please view the following:

templates/package/snippets/package_basic_fields.html templates/group/snippets/group_form.html templates/organization/snippets/organization_form.html

There are many ways to handle this, the simplest would probably be http://jinja.pocoo.org/docs/2.9/templates/#replace

Managed to change the suffix but can't fine a way to change the suffix of the domain.

Doing |replace("group, "theme) on L15 in group/snippets/group_form.html should replace the "/group" displayed in the URL slug preview

6. Is it possible to purge deleted groups/themes?

Yes, please view: http://docs.ckan.org/en/ckan-2.6.3/api/index.html#ckan.logic.action.delete.group_purge

Tried to use ckanapi but can't fine any group, maybe because of the changes from groups to theme?

^C(vagrant) vagrant@vagrant-ubuntu-trusty-64:/vagrant$ ckanapaction group_list -j -c /etc/ckan/default/ckan.ini
[]

There is a typo, ^C(vagrant) vagrant@vagrant-ubuntu-trusty-64:/vagrant$ ckanapaction group_list -j -c /etc/ckan/default/ckan.ini

it says ckanapaction instead of ckanapiaction (missing i, I presume?)

7. Slug and icon are still wrong ones here: http://iod-ckan-live.aws.smallmedia.org.uk/dataset/groups/title

8. Themes page in Farsi crashes

http://iod-ckan-live.aws.smallmedia.org.uk/theme ✅ http://iod-ckan-live.aws.smallmedia.org.uk/fa_IR/theme ❌ Server Error. An internal server error occurred.

[riccardoerra]

Testing it and seems all good.

One thing I noticed is that if I go to themes page it shows no themes:

But if I try to add a theme to a dataset the dropdown shows some themes:

Maybe some old themes that did not get created correctly. What's the best way to purge themes?

[TheoStefou]

How would someone do the remapping in version 2.9 where IRoutes is now not called? I've seen that i should be using IBlueprint but i don't see how.