search

EUDAT-B2SHARE / b2share

B2SHARE software for the EUDAT CDI services.
https://b2share.eudat.eu
GNU General Public License v2.0
35 stars 32 forks source link

editing data with scanding letters on title #772

Closed llehtine closed 7 years ago

llehtine commented 9 years ago

if there is scanding letters in title (might also be on other fields, too), editing the deposit metadata will fail. This is error which is coming to log files:

Hello:

This exception has already been seen 1 times
    last time it was seen: 2015-09-02 14:57:51
    last time it was notified: 2015-09-02 14:57:51

* 2015-09-02 14:57:51 -> UnicodeEncodeError: 'ascii' codec can't encode character u'\u0308' in position 28: ordinal not in range(128) (edit.py:195:_bibdoc_modify_files)

** User details
                                     agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:40.0) Gecko/20100101 Firefox/40.0
                        dashboard_settings: {'orderMiddle': [u'WebSearchSettings', u'WebBasketSettings'], 'orderLeft': [u'WebAlertSettings', u'WebMessageSettings', u'WebAccountSettings'], 'orderRight': u'WebTagSettings'}
                                     email: eudat-support@csc.fi
                               family_name:
                               given_names:
                                     group: [u'linguistics_domain_administrators', u'nrm_domain_administrators', u'gbif_domain_administrators', u'generic_domain_administrators', u'euon_domain_administrators', u'rda_domain_administrators', u'drihm_domain_administrators', u'bbmri_domain_administrators']
                                     guest: 0
                                        id: 1
                              login_method: Local
                                  nickname: admin
                                      note: 1
            precached_canseehiddenmarctags: True
precached_permitted_restricted_collections: []
                    precached_sendcomments: False
                        precached_useadmin: True
                       precached_usealerts: True
                      precached_useapprove: True
                      precached_usebaskets: True
                       precached_usegroups: True
                        precached_useloans: True
                     precached_usemessages: True
             precached_usepaperattribution: True
                   precached_usepaperclaim: True
                        precached_usestats: True
                   precached_usesuperadmin: True
                   precached_viewclaimlink: False
                 precached_viewsubmissions: False
                                   referer: <https://b2share.eudat.eu/b2deposit/251/edit>
                               remote_host:
                                 remote_ip: 193.166.2.127
                                   session: None
                                  settings: {'login_method': 'Local', 'dashboard_settings': {'orderMiddle': [u'WebSearchSettings', u'WebBasketSettings'], 'orderLeft': [u'WebAlertSettings', u'WebMessageSettings', u'WebAccountSettings'], 'orderRight': u'WebTagSettings'}}
                                       uid: 1
                                       uri: </b2deposit/251/update>

** Traceback details

Traceback (most recent call last):
  File "/var/www/.virtualenvs/b2share/lib/python2.7/site-packages/invenio-2.0.7.dev20150901-py2.7.egg/invenio/ext/legacy/__init__.py", line 124, in __call__
    response = self.app.full_dispatch_request()
  File "/var/www/.virtualenvs/b2share/lib/python2.7/site-packages/flask/app.py", line 1477, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/var/www/.virtualenvs/b2share/lib/python2.7/site-packages/flask_restful/__init__.py", line 258, in error_router
    return original_handler(e)
  File "/var/www/.virtualenvs/b2share/lib/python2.7/site-packages/invenio-2.0.7.dev20150901-py2.7.egg/invenio/base/wrappers.py", line 125, in handle_user_exception
    return super(Flask, self).handle_user_exception(e)
  File "/var/www/.virtualenvs/b2share/lib/python2.7/site-packages/flask/app.py", line 1381, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/var/www/.virtualenvs/b2share/lib/python2.7/site-packages/flask/app.py", line 1475, in full_dispatch_request
    rv = self.dispatch_request()
  File "/var/www/.virtualenvs/b2share/lib/python2.7/site-packages/flask/app.py", line 1461, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/var/www/.virtualenvs/b2share/lib/python2.7/site-packages/flask_login.py", line 658, in decorated_view
    return func(*args, **kwargs)
  File "/var/www/.virtualenvs/b2share/lib/python2.7/site-packages/invenio-2.0.7.dev20150901-py2.7.egg/invenio/b2share/modules/b2deposit/views.py", line 94, in updatemeta
    return edt.update(int(recid), request.form)
  File "/var/www/.virtualenvs/b2share/lib/python2.7/site-packages/invenio-2.0.7.dev20150901-py2.7.egg/invenio/b2share/modules/b2deposit/edit.py", line 61, in update
    _bibdoc_modify_files(recid, form)
  File "/var/www/.virtualenvs/b2share/lib/python2.7/site-packages/invenio-2.0.7.dev20150901-py2.7.egg/invenio/b2share/modules/b2deposit/edit.py", line 195, in _bibdoc_modify_files
    current_app.logger.info("renaming bibdoc/file: {}/'{}' -> '{}'".format(recid, docname, newname))
UnicodeEncodeError: 'ascii' codec can't encode character u'\u0308' in position 28: ordinal not in range(128)

** Stack frame details

Frame _bibdoc_modify_files in /var/www/.virtualenvs/b2share/lib/python2.7/site-packages/invenio-2.0.7.dev20150901-py2.7.egg/invenio/b2share/modules/b2deposit/edit.py at line 195
                       docid =  '2446'
                        form =  "ImmutableMultiDict([('csrf_token', u'None'), ('description', u'A test file to asses the feasibility of b2share for different purposes.'), ('contributors', u''), ('creator', u'Urpo Kaila'), ('contact_email', u'ukk@iki.fi'), ('licence', u'Creative Commons Attribution-NonCommercial-NoDerivs (CC-BY-NC-ND)'), ('__file__name__2446', u'Daniel ajaa moottoriveneella\\u0308'), ('keywords', u'Test'), ('publication_date', u'2015-09-02'), ('discipline', u'Ancient history'), ('publisher', u'http://b2share.eu [...]
                     newname =  "u'Daniel ajaa moottoriveneella\\u0308'"
                           k =  "'embargo_till'"
                         act =  "'rename'"
                     actions =  "{2446: ('rename', 'Daniel ajaa moottoriveneella\\xcc\\x88', u'Daniel ajaa moottoriveneella\\u0308')}"
                  BibRecDocs =  "<class 'invenio.legacy.bibdocfile.api.BibRecDocs'>"
                       recid =  '251'
                           v =  "u'5555234523452345234523452345'"
                     docname =  "'Daniel ajaa moottoriveneella\\xcc\\x88'"
                     recdocs =  'BibRecDocs(251)'
                           _ =  '2446'

Frame update in /var/www/.virtualenvs/b2share/lib/python2.7/site-packages/invenio-2.0.7.dev20150901-py2.7.egg/invenio/b2share/modules/b2deposit/edit.py at line 61
-------------------------------------------------------------------------------
        58     if meta_form.validate_on_submit():
        59         current_app.logger.info("Updating record {}".format(recid))
        60
---->   61         _bibdoc_modify_files(recid, form)
        62
        63         rec_changes = {}
        64         add_basic_fields(rec_changes, form, meta)
-------------------------------------------------------------------------------
                   meta_form =  '<wtforms.ext.sqlalchemy.orm.genericMetadataForm object at 0x7f140f921310>'
                      domain =  "'generic'"
                        form =  "ImmutableMultiDict([('csrf_token', u'None'), ('description', u'A test file to asses the feasibility of b2share for different purposes.'), ('contributors', u''), ('creator', u'Urpo Kaila'), ('contact_email', u'ukk@iki.fi'), ('licence', u'Creative Commons Attribution-NonCommercial-NoDerivs (CC-BY-NC-ND)'), ('__file__name__2446', u'Daniel ajaa moottoriveneella\\u0308'), ('keywords', u'Test'), ('publication_date', u'2015-09-02'), ('discipline', u'Ancient history'), ('publisher', u'http://b2share.eu [...]
            bibformat_engine =  "<module 'invenio.modules.formatter.engine' from '/var/www/.virtualenvs/b2share/lib/python2.7/site-packages/invenio-2.0.7.dev20150901-py2.7.egg/invenio/modules/formatter/engine.pyc'>"
                         bfo =  '<invenio.modules.formatter.engine.BibFormatObject object at 0x7f140f705990>'
                  get_record =  '<function get_record at 0x7f140cc96230>'
                        meta =  '<SubmissionMetadata None>'
                       recid =  '251'
                   bibupload =  '<function bibupload at 0x7f140e9925f0>'
                   metaclass =  "<class 'flask_sqlalchemy.genericMetadata'>"

Frame updatemeta in /var/www/.virtualenvs/b2share/lib/python2.7/site-packages/invenio-2.0.7.dev20150901-py2.7.egg/invenio/b2share/modules/b2deposit/views.py at line 94
-------------------------------------------------------------------------------
        91 @blueprint.route('/<recid>/update', methods=['POST'])
        92 @login_required
        93 def updatemeta(recid):
---->   94     return edt.update(int(recid), request.form)
-------------------------------------------------------------------------------
                       recid =  "u'251'"

Frame decorated_view in /var/www/.virtualenvs/b2share/lib/python2.7/site-packages/flask_login.py at line 658
-------------------------------------------------------------------------------
       655             return func(*args, **kwargs)
       656         elif not current_user.is_authenticated():
       657             return current_app.login_manager.unauthorized()
---->  658         return func(*args, **kwargs)
       659     return decorated_view
       660
       661
-------------------------------------------------------------------------------
                        args =  '()'
                        func =  '<function updatemeta at 0x7f1437406398>'
                      kwargs =  "{'recid': u'251'}"

Frame dispatch_request in /var/www/.virtualenvs/b2share/lib/python2.7/site-packages/flask/app.py at line 1461
-------------------------------------------------------------------------------
      1458            and req.method == 'OPTIONS':
      1459             return self.make_default_options_response()
      1460         # otherwise dispatch to the handler for that endpoint
----> 1461         return self.view_functions[rule.endpoint](**req.view_args)
      1462
      1463     def full_dispatch_request(self):
      1464         """Dispatches the request and on top of that performs request
-------------------------------------------------------------------------------
                        self =  "<Flask 'invenio.base'>"
                         req =  "<LegacyRequest 'https://b2share.eudat.eu/b2deposit/251/update' [POST]>"
                        rule =  "<Rule '/b2deposit/<recid>/update' (POST, OPTIONS) -> b2deposit.updatemeta>"

Frame full_dispatch_request in /var/www/.virtualenvs/b2share/lib/python2.7/site-packages/flask/app.py at line 1477
-------------------------------------------------------------------------------
      1474             if rv is None:
      1475                 rv = self.dispatch_request()
      1476         except Exception as e:
----> 1477             rv = self.handle_user_exception(e)
      1478         response = self.make_response(rv)
      1479         response = self.process_response(response)
      1480         request_finished.send(self, response=response)
-------------------------------------------------------------------------------
                          rv =  'None'
                           e =  "UnicodeEncodeError('ascii', u'Daniel ajaa moottoriveneella\\u0308', 28, 29, 'ordinal not in range(128)')"
                        self =  "<Flask 'invenio.base'>"

Frame handle_user_exception in /var/www/.virtualenvs/b2share/lib/python2.7/site-packages/flask/app.py at line 1381
-------------------------------------------------------------------------------
      1378             if isinstance(e, typecheck):
      1379                 return handler(e)
      1380
----> 1381         reraise(exc_type, exc_value, tb)
      1382
      1383     def handle_exception(self, e):
      1384         """Default exception handling that kicks in when an exception
-------------------------------------------------------------------------------
                           e =  "UnicodeEncodeError('ascii', u'Daniel ajaa moottoriveneella\\u0308', 28, 29, 'ordinal not in range(128)')"
                    handlers =  'None'
                        self =  "<Flask 'invenio.base'>"
                   exc_value =  "UnicodeEncodeError('ascii', u'Daniel ajaa moottoriveneella\\u0308', 28, 29, 'ordinal not in range(128)')"
          blueprint_handlers =  '()'
                app_handlers =  '()'
                          tb =  '<traceback object at 0x7f140f92b560>'
                    exc_type =  "<type 'exceptions.UnicodeEncodeError'>"

Frame handle_user_exception in /var/www/.virtualenvs/b2share/lib/python2.7/site-packages/invenio-2.0.7.dev20150901-py2.7.egg/invenio/base/wrappers.py at line 125
-------------------------------------------------------------------------------
       122         Hook a before handle to it.
       123         """
       124         before_handle_user_exception.send(e)
---->  125         return super(Flask, self).handle_user_exception(e)
       126
       127
       128 def lazy_import(name):
-------------------------------------------------------------------------------
                        self =  "<Flask 'invenio.base'>"
                           e =  "UnicodeEncodeError('ascii', u'Daniel ajaa moottoriveneella\\u0308', 28, 29, 'ordinal not in range(128)')"

Frame error_router in /var/www/.virtualenvs/b2share/lib/python2.7/site-packages/flask_restful/__init__.py at line 258
-------------------------------------------------------------------------------
       255         """
       256         if self._has_fr_route():
       257             return self.handle_error(e)
---->  258         return original_handler(e)
       259
       260     def handle_error(self, e):
       261         """Error handler for the API transforms a raised exception into a Flask
-------------------------------------------------------------------------------
            original_handler =  "<bound method Flask.handle_user_exception of <Flask 'invenio.base'>>"
                        self =  '<flask_restful.Api object at 0x7f1436cda7d0>'
                           e =  "UnicodeEncodeError('ascii', u'Daniel ajaa moottoriveneella\\u0308', 28, 29, 'ordinal not in range(128)')"

Frame full_dispatch_request in /var/www/.virtualenvs/b2share/lib/python2.7/site-packages/flask/app.py at line 1477
-------------------------------------------------------------------------------
      1474             if rv is None:
      1475                 rv = self.dispatch_request()
      1476         except Exception as e:
----> 1477             rv = self.handle_user_exception(e)
      1478         response = self.make_response(rv)
      1479         response = self.process_response(response)
      1480         request_finished.send(self, response=response)
-------------------------------------------------------------------------------
                          rv =  'None'
                           e =  "UnicodeEncodeError('ascii', u'Daniel ajaa moottoriveneella\\u0308', 28, 29, 'ordinal not in range(128)')"
                        self =  "<Flask 'invenio.base'>"

Frame __call__ in /var/www/.virtualenvs/b2share/lib/python2.7/site-packages/invenio-2.0.7.dev20150901-py2.7.egg/invenio/ext/legacy/__init__.py at line 127
-------------------------------------------------------------------------------
       124                     response = self.app.full_dispatch_request()
       125                 except Exception as e:
       126                     from invenio.ext.logging import register_exception
---->  127                     register_exception(req=request, alert_admin=True)
       128                     response = self.app.handle_exception(e)
       129
       130                 return response(environ, start_response)
-------------------------------------------------------------------------------
              start_response =  '<built-in method start_response of mod_wsgi.Adapter object at 0x7f140f78de40>'
                     environ =  "{'wsgi.version': (1, 0), 'mod_wsgi.listener_port': '443', 'HTTP_REFERER': 'https://b2share.eudat.eu/b2deposit/251/edit', 'mod_wsgi.listener_host': '', 'SERVER_SOFTWARE': 'Apache', 'SCRIPT_NAME': '', 'mod_wsgi.enable_sendfile': '0', 'mod_wsgi.handler_script': '', 'SERVER_SIGNATURE': '', 'REQUEST_METHOD': 'POST', 'PATH_INFO': '/b2deposit/251/update', 'SERVER_PROTOCOL': 'HTTP/1.1', 'QUERY_STRING': '', 'SSL_TLS_SNI': 'b2share.eudat.eu', 'CONTENT_LENGTH': '507', 'HTTP_USER_AGENT': 'Mozilla/5.0 (Maci [...]
                        self =  '<invenio.ext.legacy.LegacyAppMiddleware object at 0x7f14331e1d50>'
          register_exception =  '<function register_exception at 0x7f1436920ed8>'
                           e =  "UnicodeEncodeError('ascii', u'Daniel ajaa moottoriveneella\\u0308', 28, 29, 'ordinal not in range(128)')"```
emanueldima commented 7 years ago

won't fix in v1, solved in v2