Crashes if unicode characters occur

[nesnera]

How to reproduce: Crate new page with unicode characters in a title e.g. "Otevřená města" CKAN "2.7.0"

  extra_vars={'dataset_type': package_type})
File '/usr/lib/ckan/default/src/ckan/ckan/lib/base.py', line 177 in render
  return cached_template(template_name, render_template)
File '/usr/lib/ckan/default/lib/python2.7/site-packages/pylons/templating.py', line 249 in cached_template
  return render_func()
File '/usr/lib/ckan/default/src/ckan/ckan/lib/base.py', line 131 in render_template
  return render_jinja2(template_name, globs)
File '/usr/lib/ckan/default/src/ckan/ckan/lib/base.py', line 88 in render_jinja2
  return template.render(**extra_vars)
File '/usr/lib/ckan/default/lib/python2.7/site-packages/jinja2/environment.py', line 989 in render
  return self.environment.handle_exception(exc_info, True)
File '/usr/lib/ckan/default/lib/python2.7/site-packages/jinja2/environment.py', line 754 in handle_exception
  reraise(exc_type, exc_value, tb)
File '/usr/lib/ckan/default/src/ckan/ckan/templates/package/search.html', line 2 in top-level template code
  {% import 'macros/form.html' as form %}
File '/usr/lib/ckan/default/src/ckan/ckan/templates/page.html', line 1 in top-level template code
  {% extends "base.html" %}
File '/usr/lib/ckan/default/src/ckanext-pages/ckanext/pages/theme/templates_main/base.html', line 1 in top-level template code
  {% ckan_extends %}
File '/usr/lib/ckan/default/src/ckan/ckan/templates/base.html', line 103 in top-level template code
  {%- block page %}{% endblock -%}
File '/usr/lib/ckan/default/src/ckan/ckan/templates/page.html', line 14 in block "page"
  {%- block header %}
File '/usr/lib/ckan/default/src/ckan/ckan/templates/page.html', line 15 in block "header"
  {% include "header.html" %}
File '/usr/lib/ckan/default/src/ckanext-pages/ckanext/pages/theme/templates_main/header.html', line 1 in top-level template code
  {% ckan_extends %}
File '/usr/lib/ckan/default/src/ckan/ckan/templates/header.html', line 1 in top-level template code
  {% block header_wrapper %}
File '/usr/lib/ckan/default/src/ckan/ckan/templates/header.html', line 98 in block "header_wrapper"
  {% block header_site_navigation %}
File '/usr/lib/ckan/default/src/ckan/ckan/templates/header.html', line 101 in block "header_site_navigation"
  {% block header_site_navigation_tabs %}
File '/usr/lib/ckan/default/src/ckan/ckan/templates/header.html', line 102 in block "header_site_navigation_tabs"
  {{ h.build_nav_main(
File '/usr/lib/ckan/default/src/ckanext-pages/ckanext/pages/plugin.py', line 53 in build_pages_nav_main
  link = h.literal('<a href="/pages/%s">%s</a>' % (str(page['name']), str(page['title'])))
UnicodeEncodeError: 'ascii' codec can't encode character u'\u0159' in position 4: ordinal not in range(128)```

[torfsen]

Judging from the stacktrace, that seems to be a problem in the ckanext-pages extension. Please check if the problem persists if you disable that extension. If it doesn't, please create a bug for ckanext-pages instead and make to sure to mention which version of the plugin you are using. Thanks!

[metaodi]

@torfsen this is the ckanext-pages repository 😂

[torfsen]

@metaodi Argh. That's what happens when your head is already in week-end mode. Sorry @nesnera for the confusion!

[torfsen]

@nesnera I cannot reproduce the problem on CKAN 2.7.0 running the latest ckanext-pages release 0.1.0. What version of ckanext-pages are you running? Does the error still occur if you disable all other plugins?

[nesnera]

@torfsen CLI setup output:

$ pip install -e 'git+https://github.com/ckan/ckanext-pages.git#egg=ckanext-pages'
Obtaining ckanext-pages from git+https://github.com/ckan/ckanext-pages.git#egg=ckanext-pages
  Cloning https://github.com/ckan/ckanext-pages.git to ./ckanext-pages
  Running setup.py egg_info for package ckanext-pages
Installing collected packages: ckanext-pages
  Running setup.py develop for ckanext-pages
    Installing /usr/lib/ckan/default/lib/python2.7/site-packages/ckanext-pages-nspkg.pth
    Creating /usr/lib/ckan/default/lib/python2.7/site-packages/ckanext-pages.egg-link (link to .)
    Removing ckanext-pages 0.1.0 from easy-install.pth file
    Adding ckanext-pages 0.1.0 to easy-install.pth file

    Installed /usr/lib/ckan/default/src/ckanext-pages
Successfully installed ckanext-pages
Cleaning up... 

Yes, I tried it with "vanilla" CKAN 2.7.0 and disabled all other plugins. And what is worst - if the plugin is activated the web application doesn't work at all (with error above). API is intact. (e.g. http://ckandevel.otevrenamesta.cz/api/util/status) Please, try create page named "Otevřená města" as item in the menu.

[torfsen]

Finally found time to take another look at this. I can now reproduce the issue with CKAN 2.7 and ckanext-pages 0.1.0. Simply creating a page with a Unicode name still works. However, adding the page to the menu does trigger the error (assuming the page's visibility is "Public"). The problems occurs both for normal pages and for blog posts.