Sieve filters doesn't work

[baptcottey]

Hi,

On my modoboa siever filetrs don't work, i get 500 Internal Error. I turned on Debug and i get this error on this URL /sfilters/Filtre/?_=1469616090572

UnicodeEncodeError at /sfilters/Filtre/
'ascii' codec can't encode character u'\xe9' in position 739: ordinal not in range(128)
Request Method: GET
Request URL:    https://mx.xx.fr/sfilters/Filtre/?_=1469616090572
Django Version: 1.9.8
Exception Type: UnicodeEncodeError
Exception Value:    
'ascii' codec can't encode character u'\xe9' in position 739: ordinal not in range(128)
Exception Location: /srv/modoboa/env/local/lib/python2.7/site-packages/modoboa_sievefilters/views.py in getfs, line 103
Python Executable:  /usr/bin/uwsgi-core
Python Version: 2.7.9
Python Path:    
['.',
 '',
 '/srv/modoboa/env/lib/python2.7',
 '/srv/modoboa/env/lib/python2.7/plat-arm-linux-gnueabihf',
 '/srv/modoboa/env/lib/python2.7/lib-tk',
 '/srv/modoboa/env/lib/python2.7/lib-old',
 '/srv/modoboa/env/lib/python2.7/lib-dynload',
 '/usr/lib/python2.7',
 '/usr/lib/python2.7/plat-arm-linux-gnueabihf',
 '/usr/lib/python2.7/lib-tk',
 '/srv/modoboa/env/local/lib/python2.7/site-packages']
Server time:    mer, 27 Jul 2016 12:43:31 +0200
Unicode error hint

The string that could not be encoded/decoded was: /i> Télécha
Traceback Switch to copy-and-paste view

/srv/modoboa/env/local/lib/python2.7/site-packages/django/core/handlers/base.py in get_response
                    response = self.process_exception_by_middleware(e, request) ...
▶ Local vars
/srv/modoboa/env/local/lib/python2.7/site-packages/django/core/handlers/base.py in get_response
                    response = wrapped_callback(request, *callback_args, **callback_kwargs) ...
▶ Local vars
/srv/modoboa/env/local/lib/python2.7/site-packages/django/utils/decorators.py in inner
                    return func(*args, **kwargs) ...
▶ Local vars
/srv/modoboa/env/local/lib/python2.7/site-packages/django/utils/decorators.py in inner
                    return func(*args, **kwargs) ...
▶ Local vars
/srv/modoboa/env/local/lib/python2.7/site-packages/django/contrib/auth/decorators.py in _wrapped_view
                return view_func(request, *args, **kwargs) ...
▶ Local vars
/srv/modoboa/env/local/lib/python2.7/site-packages/modoboa/admin/lib.py in wrapped_f
                return f(request, *args, **kwargs) ...
▶ Local vars
/srv/modoboa/env/local/lib/python2.7/site-packages/modoboa_sievefilters/views.py in getfs
            _("Actions"), fset_menu(editormode, name) ...
▶ Local vars
Request information

GET
Variable    Value
_   
u'1469616090572'
POST
No POST data
FILES
No FILES data
COOKIES
Variable    Value
csrftoken   
'aL2rdOHJayk2yfxUMXD3ndPpvVeKCZxD'
sessionid   
'uhl87rd3v40palbyxmes9h7wjgzydcw6'
_ga 
'GA1.2.1586104241.1419605983'
META
Variable    Value
wsgi.multiprocess   
True
HTTP_COOKIE 
'_ga=GA1.2.1586104241.1419605983; sessionid=uhl87rd3v40palbyxmes9h7wjgzydcw6; csrftoken=aL2rdOHJayk2yfxUMXD3ndPpvVeKCZxD'
UWSGI_SCRIPT    
'instance.wsgi:application'
SCRIPT_NAME 
u''
REQUEST_METHOD  
'GET'
PATH_INFO   
u'/sfilters/Filtre/'
SERVER_PROTOCOL 
'HTTP/1.1'
QUERY_STRING    
'_=1469616090572'
CONTENT_LENGTH  
''
HTTP_USER_AGENT 
'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
HTTP_CONNECTION 
'keep-alive'
SERVER_NAME 
'mx.xx.fr'
REMOTE_ADDR 
'193.248.177.230'
wsgi.url_scheme 
'https'
SERVER_PORT 
'443'
uwsgi.node  
'scw-71a319'
DOCUMENT_ROOT   
'/srv/modoboa/instance'
wsgi.input  
<uwsgi._Input object at 0xb4992cc0>
HTTP_DNT    
'1'
HTTP_HOST   
'mx.xx.fr'
HTTPS   
'on'
wsgi.multithread    
False
HTTP_UPGRADE_INSECURE_REQUESTS  
'1'
HTTP_CACHE_CONTROL  
'max-age=0'
REQUEST_URI 
'/sfilters/Filtre/?_=1469616090572'
HTTP_ACCEPT 
'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8'
wsgi.version    
(1, 0)
wsgi.run_once   
False
wsgi.errors 
<open file 'wsgi_errors', mode 'w' at 0xb51bec80>
REMOTE_PORT 
'58672'
HTTP_ACCEPT_LANGUAGE    
'fr-FR,fr;q=0.8,en-US;q=0.6,en;q=0.4'
UWSGI_APPID 
'mx.xx.fr|'
uwsgi.version   
'2.0.7-debian'
CONTENT_TYPE    
''
wsgi.file_wrapper   
''
CSRF_COOKIE 
u'aL2rdOHJayk2yfxUMXD3ndPpvVeKCZxD'
HTTP_ACCEPT_ENCODING    
'gzip, deflate, sdch, br'
Settings
Using settings module instance.settings
Setting Value
SECURE_REDIRECT_EXEMPT  
[]
SECURE_BROWSER_XSS_FILTER   
False
USE_X_FORWARDED_PORT    
False
USE_THOUSAND_SEPARATOR  
False
CSRF_COOKIE_SECURE  
False
LANGUAGE_CODE   
'en-us'
ROOT_URLCONF    
'instance.urls'
MANAGERS    
[]
BASE_DIR    
'.'
SILENCED_SYSTEM_CHECKS  
['fields.W342']
DEFAULT_CHARSET 
'utf-8'
SWAGGER_SETTINGS    
{'api_key': '',
 'api_path': '/',
 'api_version': '1.0',
 'doc_expansion': 'none',
 'enabled_methods': ['get', 'post', 'put', 'patch', 'delete'],
 'exclude_namespaces': [],
 'info': {'contact': 'contact@modoboa.com',
          'description': 'Modoboa API, requires a valid token.',
          'title': 'Modoboa API'},
 'is_authenticated': False,
 'is_superuser': False,
 'permission_denied_handler': None,
 'resource_access_handler': None,
 'template_path': 'rest_framework_swagger/index.html',
 'token_type': 'Token',
 'unauthenticated_user': 'django.contrib.auth.models.AnonymousUser'}
SESSION_SERIALIZER  
'django.contrib.sessions.serializers.JSONSerializer'
STATIC_ROOT 
'./sitestatic'
ALLOWED_HOSTS   
['mx.xx.fr']
MESSAGE_STORAGE 
'django.contrib.messages.storage.fallback.FallbackStorage'
EMAIL_SUBJECT_PREFIX    
'[Django] '
SERVER_EMAIL    
'root@localhost'
PASSWORD_MIN_LENGTH 
u'********************'
SECURE_HSTS_SECONDS 
0
STATICFILES_FINDERS 
['django.contrib.staticfiles.finders.FileSystemFinder',
 'django.contrib.staticfiles.finders.AppDirectoriesFinder']
SESSION_CACHE_ALIAS 
'default'
SESSION_COOKIE_DOMAIN   
None
SESSION_COOKIE_NAME 
'sessionid'
TIME_INPUT_FORMATS  
['%H:%M:%S', '%H:%M:%S.%f', '%H:%M']
MODOBOA_APPS    
('modoboa',
 'modoboa.core',
 'modoboa.lib',
 'modoboa.admin',
 'modoboa.relaydomains',
 'modoboa.limits',
 'modoboa_amavis',
 'modoboa_pdfcredentials',
 'modoboa_postfix_autoreply',
 'modoboa_sievefilters',
 'modoboa_stats',
 'modoboa_webmail')
DATABASES   
{'amavis': {'ATOMIC_REQUESTS': True,
            'AUTOCOMMIT': True,
            'CONN_MAX_AGE': 0,
            'ENGINE': 'django.db.backends.postgresql_psycopg2',
            'HOST': '127.0.0.1',
            'NAME': 'amavis',
            'OPTIONS': {},
            'PASSWORD': u'********************',
            'PORT': '',
            'TEST': {'CHARSET': None,
                     'COLLATION': None,
                     'MIRROR': None,
                     'NAME': None},
            'TIME_ZONE': None,
            'USER': 'amavis'},
 'default': {'ATOMIC_REQUESTS': True,
             'AUTOCOMMIT': True,
             'CONN_MAX_AGE': 0,
             'ENGINE': 'django.db.backends.postgresql_psycopg2',
             'HOST': '127.0.0.1',
             'NAME': 'modoboa',
             'OPTIONS': {},
             'PASSWORD': u'********************',
             'PORT': '',
             'TEST': {'CHARSET': None,
                      'COLLATION': None,
                      'MIRROR': None,
                      'NAME': None},
             'TIME_ZONE': None,
             'USER': 'modoboa'}}
EMAIL_SSL_KEYFILE   
u'********************'
FILE_UPLOAD_DIRECTORY_PERMISSIONS   
None
PASSWORD_COMPLEXITY 
u'********************'
FILE_UPLOAD_PERMISSIONS 
None
FILE_UPLOAD_HANDLERS    
['django.core.files.uploadhandler.MemoryFileUploadHandler',
 'django.core.files.uploadhandler.TemporaryFileUploadHandler']
DEFAULT_CONTENT_TYPE    
'text/html'
APPEND_SLASH    
True
LOCALE_PATHS    
[]
DATABASE_ROUTERS    
['modoboa_amavis.dbrouter.AmavisRouter']
DEFAULT_TABLESPACE  
''
YEAR_MONTH_FORMAT   
'F Y'
STATICFILES_STORAGE 
'django.contrib.staticfiles.storage.StaticFilesStorage'
CACHES  
{'default': {'BACKEND': 'django.core.cache.backends.locmem.LocMemCache'}}
SESSION_COOKIE_PATH 
'/'
EMAIL_HOST_PASSWORD 
u'********************'
SECURE_CONTENT_TYPE_NOSNIFF 
False
MIDDLEWARE_CLASSES  
('x_forwarded_for.middleware.XForwardedForMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.middleware.common.CommonMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware',
 'django.middleware.locale.LocaleMiddleware',
 'django.middleware.clickjacking.XFrameOptionsMiddleware',
 'modoboa.lib.middleware.AjaxLoginRedirect',
 'modoboa.lib.middleware.CommonExceptionCatcher',
 'modoboa.lib.middleware.RequestCatcherMiddleware')
USE_I18N    
True
THOUSAND_SEPARATOR  
','
SECRET_KEY  
u'********************'
MODOBOA_API_URL 
u'********************'
LANGUAGE_COOKIE_NAME    
'django_language'
DEFAULT_INDEX_TABLESPACE    
''
LOGGING_CONFIG  
'logging.config.dictConfig'
TEMPLATE_LOADERS    
['django.template.loaders.filesystem.Loader',
 'django.template.loaders.app_directories.Loader']
FIRST_DAY_OF_WEEK   
0
WSGI_APPLICATION    
'instance.wsgi.application'
TEMPLATE_DEBUG  
False
X_FRAME_OPTIONS 
'SAMEORIGIN'
CSRF_COOKIE_NAME    
'csrftoken'
FORCE_SCRIPT_NAME   
None
USE_X_FORWARDED_HOST    
False
CSRF_HEADER_NAME    
'HTTP_X_CSRFTOKEN'
EMAIL_TIMEOUT   
None
SECURE_SSL_HOST 
None
SIGNING_BACKEND 
'django.core.signing.TimestampSigner'
SESSION_COOKIE_SECURE   
False
CSRF_COOKIE_DOMAIN  
None
FILE_CHARSET    
'utf-8'
DEBUG   
True
CSRF_TRUSTED_ORIGINS    
[]
LANGUAGE_COOKIE_DOMAIN  
None
DEFAULT_FILE_STORAGE    
'django.core.files.storage.FileSystemStorage'
INSTALLED_APPS  
('django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.messages',
 'django.contrib.sites',
 'django.contrib.staticfiles',
 'reversion',
 'rest_framework.authtoken',
 'rest_framework_swagger',
 'modoboa',
 'modoboa.core',
 'modoboa.lib',
 'modoboa.admin',
 'modoboa.relaydomains',
 'modoboa.limits',
 'modoboa_amavis',
 'modoboa_pdfcredentials',
 'modoboa_postfix_autoreply',
 'modoboa_sievefilters',
 'modoboa_stats',
 'modoboa_webmail')
LANGUAGES   
[('af', 'Afrikaans'),
 ('ar', 'Arabic'),
 ('ast', 'Asturian'),
 ('az', 'Azerbaijani'),
 ('bg', 'Bulgarian'),
 ('be', 'Belarusian'),
 ('bn', 'Bengali'),
 ('br', 'Breton'),
 ('bs', 'Bosnian'),
 ('ca', 'Catalan'),
 ('cs', 'Czech'),
 ('cy', 'Welsh'),
 ('da', 'Danish'),
 ('de', 'German'),
 ('el', 'Greek'),
 ('en', 'English'),
 ('en-au', 'Australian English'),
 ('en-gb', 'British English'),
 ('eo', 'Esperanto'),
 ('es', 'Spanish'),
 ('es-ar', 'Argentinian Spanish'),
 ('es-co', 'Colombian Spanish'),
 ('es-mx', 'Mexican Spanish'),
 ('es-ni', 'Nicaraguan Spanish'),
 ('es-ve', 'Venezuelan Spanish'),
 ('et', 'Estonian'),
 ('eu', 'Basque'),
 ('fa', 'Persian'),
 ('fi', 'Finnish'),
 ('fr', 'French'),
 ('fy', 'Frisian'),
 ('ga', 'Irish'),
 ('gd', 'Scottish Gaelic'),
 ('gl', 'Galician'),
 ('he', 'Hebrew'),
 ('hi', 'Hindi'),
 ('hr', 'Croatian'),
 ('hu', 'Hungarian'),
 ('ia', 'Interlingua'),
 ('id', 'Indonesian'),
 ('io', 'Ido'),
 ('is', 'Icelandic'),
 ('it', 'Italian'),
 ('ja', 'Japanese'),
 ('ka', 'Georgian'),
 ('kk', 'Kazakh'),
 ('km', 'Khmer'),
 ('kn', 'Kannada'),
 ('ko', 'Korean'),
 ('lb', 'Luxembourgish'),
 ('lt', 'Lithuanian'),
 ('lv', 'Latvian'),
 ('mk', 'Macedonian'),
 ('ml', 'Malayalam'),
 ('mn', 'Mongolian'),
 ('mr', 'Marathi'),
 ('my', 'Burmese'),
 ('nb', 'Norwegian Bokmal'),
 ('ne', 'Nepali'),
 ('nl', 'Dutch'),
 ('nn', 'Norwegian Nynorsk'),
 ('os', 'Ossetic'),
 ('pa', 'Punjabi'),
 ('pl', 'Polish'),
 ('pt', 'Portuguese'),
 ('pt-br', 'Brazilian Portuguese'),
 ('ro', 'Romanian'),
 ('ru', 'Russian'),
 ('sk', 'Slovak'),
 ('sl', 'Slovenian'),
 ('sq', 'Albanian'),
 ('sr', 'Serbian'),
 ('sr-latn', 'Serbian Latin'),
 ('sv', 'Swedish'),
 ('sw', 'Swahili'),
 ('ta', 'Tamil'),
 ('te', 'Telugu'),
 ('th', 'Thai'),
 ('tr', 'Turkish'),
 ('tt', 'Tatar'),
 ('udm', 'Udmurt'),
 ('uk', 'Ukrainian'),
 ('ur', 'Urdu'),
 ('vi', 'Vietnamese'),
 ('zh-hans', 'Simplified Chinese'),
 ('zh-hant', 'Traditional Chinese')]
USE_L10N    
False
SECURE_HSTS_INCLUDE_SUBDOMAINS  
False
STATICFILES_DIRS    
('/srv/modoboa/env/lib/python2.7/site-packages/modoboa/bower_components',)
PREPEND_WWW 
False
SECURE_PROXY_SSL_HEADER 
None
LANGUAGE_COOKIE_AGE 
None
SESSION_COOKIE_HTTPONLY 
True
DEBUG_PROPAGATE_EXCEPTIONS  
False
CSRF_COOKIE_AGE 
31449600
MONTH_DAY_FORMAT    
'F j'
LOGIN_URL   
'/accounts/login/'
SESSION_EXPIRE_AT_BROWSER_CLOSE 
False
TIME_FORMAT 
'P'
AUTH_USER_MODEL 
'core.User'
DATE_INPUT_FORMATS  
['%Y-%m-%d',
 '%m/%d/%Y',
 '%m/%d/%y',
 '%b %d %Y',
 '%b %d, %Y',
 '%d %b %Y',
 '%d %b, %Y',
 '%B %d %Y',
 '%B %d, %Y',
 '%d %B %Y',
 '%d %B, %Y']
AUTHENTICATION_BACKENDS 
('modoboa.lib.authbackends.SimpleBackend',)
AUTH_PASSWORD_VALIDATORS    
u'********************'
PASSWORD_RESET_TIMEOUT_DAYS 
u'********************'
SESSION_FILE_PATH   
None
CACHE_MIDDLEWARE_ALIAS  
'default'
SESSION_SAVE_EVERY_REQUEST  
False
NUMBER_GROUPING 
0
SESSION_ENGINE  
'django.contrib.sessions.backends.db'
CSRF_FAILURE_VIEW   
'django.views.csrf.csrf_failure'
CSRF_COOKIE_PATH    
'/'
LOGIN_REDIRECT_URL  
'/accounts/profile/'
DECIMAL_SEPARATOR   
'.'
IGNORABLE_404_URLS  
[]
MIGRATION_MODULES   
{}
TEMPLATE_STRING_IF_INVALID  
''
LOGOUT_URL  
'/accounts/logout/'
EMAIL_USE_TLS   
False
FIXTURE_DIRS    
[]
EMAIL_HOST  
'localhost'
DATE_FORMAT 
'N j, Y'
MEDIA_ROOT  
'./media'
DEFAULT_EXCEPTION_REPORTER_FILTER   
'django.views.debug.SafeExceptionReporterFilter'
ADMINS  
[]
FORMAT_MODULE_PATH  
None
DEFAULT_FROM_EMAIL  
'webmaster@localhost'
REST_FRAMEWORK  
{'DEFAULT_AUTHENTICATION_CLASSES': ('rest_framework.authentication.TokenAuthentication',),
 'DEFAULT_PERMISSION_CLASSES': ('rest_framework.permissions.IsAuthenticated',)}
MEDIA_URL   
'/media/'
DATETIME_FORMAT 
'N j, Y, P'
TEMPLATE_DIRS   
[]
SITE_ID 
1
DISALLOWED_USER_AGENTS  
[]
ALLOWED_INCLUDE_ROOTS   
[]
LOGGING 
{'formatters': {'syslog': {'format': '%(name)s: %(levelname)s %(message)s'}},
 'handlers': {'modoboa': {'class': 'modoboa.core.loggers.SQLHandler'},
              'syslog-auth': {'class': 'logging.handlers.SysLogHandler',
                              'facility': 4,
                              'formatter': 'syslog'}},
 'loggers': {'modoboa.admin': {'handlers': ['modoboa'],
                               'level': 'INFO',
                               'propagate': False},
             'modoboa.auth': {'handlers': ['syslog-auth', 'modoboa'],
                              'level': 'INFO',
                              'propagate': False}},
 'version': 1}
SHORT_DATE_FORMAT   
'm/d/Y'
TEMPLATES   
[{'APP_DIRS': True,
  'BACKEND': 'django.template.backends.django.DjangoTemplates',
  'DIRS': [],
  'OPTIONS': {'context_processors': ['django.template.context_processors.debug',
                                     'django.template.context_processors.request',
                                     'django.contrib.auth.context_processors.auth',
                                     'django.template.context_processors.i18n',
                                     'django.template.context_processors.media',
                                     'django.template.context_processors.static',
                                     'django.template.context_processors.tz',
                                     'django.contrib.messages.context_processors.messages',
                                     'modoboa.core.context_processors.top_notifications'],
              'debug': False}}]
TEST_RUNNER 
'django.test.runner.DiscoverRunner'
CACHE_MIDDLEWARE_KEY_PREFIX 
u'********************'
SECURE_SSL_REDIRECT 
False
TIME_ZONE   
'Europe/Paris'
FILE_UPLOAD_MAX_MEMORY_SIZE 
2621440
EMAIL_BACKEND   
'django.core.mail.backends.smtp.EmailBackend'
EMAIL_USE_SSL   
False
TEMPLATE_CONTEXT_PROCESSORS 
['django.contrib.auth.context_processors.auth',
 'django.template.context_processors.debug',
 'django.template.context_processors.i18n',
 'django.template.context_processors.media',
 'django.template.context_processors.static',
 'django.template.context_processors.tz',
 'django.contrib.messages.context_processors.messages']
SESSION_COOKIE_AGE  
1209600
SETTINGS_MODULE 
'instance.settings'
USE_ETAGS   
False
LANGUAGES_BIDI  
['he', 'ar', 'fa', 'ur']
FILE_UPLOAD_TEMP_DIR    
None
INTERNAL_IPS    
[]
STATIC_URL  
'/sitestatic/'
EMAIL_PORT  
25
USE_TZ  
True
SHORT_DATETIME_FORMAT   
'm/d/Y P'
TEST_NON_SERIALIZED_APPS    
[]
PASSWORD_HASHERS    
u'********************'
ABSOLUTE_URL_OVERRIDES  
{}
LANGUAGE_COOKIE_PATH    
'/'
CACHE_MIDDLEWARE_SECONDS    
600
EMAIL_SSL_CERTFILE  
None
CSRF_COOKIE_HTTPONLY    
False
DATETIME_INPUT_FORMATS  
['%Y-%m-%d %H:%M:%S',
 '%Y-%m-%d %H:%M:%S.%f',
 '%Y-%m-%d %H:%M',
 '%Y-%m-%d',
 '%m/%d/%Y %H:%M:%S',
 '%m/%d/%Y %H:%M:%S.%f',
 '%m/%d/%Y %H:%M',
 '%m/%d/%Y',
 '%m/%d/%y %H:%M:%S',
 '%m/%d/%y %H:%M:%S.%f',
 '%m/%d/%y %H:%M',
 '%m/%d/%y']
EMAIL_HOST_USER 
''

[baptcottey]

Up ?

[tonioo]

Hi, looks like a unicode issue... You can try to temporary remove every non ascii characters from your rule names, it should work.

[baptcottey]

Hi, i don't have any non ascii characters :

[tonioo]

@baptcottey Do you think you could try the commit I just pushed ?

[baptcottey]

Hi, stupid question but how to update only sievefilters ?

[tonioo]

You need to python application installed within your virtual env.

[baptcottey]

Sorry but i'm not very familiar with Python. How to do that ?

[tonioo]

As root:

> su - modoboa
> source env/bin/activate
> pip install -e git+https://github.com/modoboa/modoboa-sievefilters#egg=modoboa-sievefilters
> exit
> service uwsgi reload

[baptcottey]

After su - modoboa when i type source env/bin/activate i get $ source env/bin/activate -su: 1: source: not found

[baptcottey]

I did it with success, and it's working ! Thank's for the fix :+1: