search

nicfit / MishMash

Music database, etc.
GNU General Public License v3.0
12 stars 4 forks source link

unicode encoding error in sqlalchemy causing app exit #240

Open redshodan opened 6 years ago

redshodan commented 6 years ago

Syncing with unsonic/mishmash got this error which caused an application exit. It doesn't say what file it was working on when this happened, but syncing this one directory does it.

baron@dib ~/src/unsonic $ ./bin/unsonic -c foo.ini sync '/data/home/baron-data/music/Franz Ferdinand/Franz Ferdinand/'
2017-11-24 09:51:44,791 DEBUG [mishmash.database][MainThread] Checking for database 'sqlite:////home/baron/src/unsonic/build/development.sqlite'
2017-11-24 09:51:44,795 DEBUG [mishmash.database][MainThread] Connecting to database 'sqlite:////home/baron/src/unsonic/build/development.sqlite'
Syncing library 'Music': paths=['/data/home/baron-data/music/Franz Ferdinand/Franz Ferdinand/']
2017-11-24 09:51:44,802 INFO  [mishmash.commands.sync.sync][MainThread] Syncing library 'Music': paths=['/data/home/baron-data/music/Franz Ferdinand/Franz Ferdinand/']
Adding artist: Franz Ferdinand
Adding album: Franz Ferdinand
Adding track: /data/home/baron-data/music/Franz Ferdinand/Franz Ferdinand/01 Jacqueline.mp3
Updating album: Franz Ferdinand
Adding track: /data/home/baron-data/music/Franz Ferdinand/Franz Ferdinand/02 Tell Her Tonight.mp3
Updating album: Franz Ferdinand
Adding track: /data/home/baron-data/music/Franz Ferdinand/Franz Ferdinand/03 Take Me Out.mp3
Updating album: Franz Ferdinand
Adding track: /data/home/baron-data/music/Franz Ferdinand/Franz Ferdinand/05 Auf Achse.mp3
Updating album: Franz Ferdinand
Adding track: /data/home/baron-data/music/Franz Ferdinand/Franz Ferdinand/06 Cheating on You.mp3
Updating album: Franz Ferdinand
Adding track: /data/home/baron-data/music/Franz Ferdinand/Franz Ferdinand/07 This Fire.mp3
Updating album: Franz Ferdinand
Adding track: /data/home/baron-data/music/Franz Ferdinand/Franz Ferdinand/08 Darts of Pleasure.mp3
Updating album: Franz Ferdinand
Adding track: /data/home/baron-data/music/Franz Ferdinand/Franz Ferdinand/09 Michael.mp3
Updating album: Franz Ferdinand
Adding track: /data/home/baron-data/music/Franz Ferdinand/Franz Ferdinand/10 Come On Home.mp3
Updating album: Franz Ferdinand
Adding track: /data/home/baron-data/music/Franz Ferdinand/Franz Ferdinand/11 40'.mp3
Updating album: Franz Ferdinand
Adding track: /data/home/baron-data/music/Franz Ferdinand/Franz Ferdinand/Franz Ferdinand - 01 - Jacqueline.mp3
Updating album: Franz Ferdinand
Adding track: /data/home/baron-data/music/Franz Ferdinand/Franz Ferdinand/Franz Ferdinand - 02 - Tell Her Tonight.mp3
Updating album: Franz Ferdinand
Adding track: /data/home/baron-data/music/Franz Ferdinand/Franz Ferdinand/Franz Ferdinand - 03 - Take Me Out.mp3
2017-11-24 09:51:45,041 ERROR [mishmash][MainThread] 'utf-8' codec can't encode character '\udce9' in position 104: surrogates not allowed
Traceback (most recent call last):
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/MishMash-0.3b4-py3.6.egg/mishmash/__main__.py", line 53, in main
    retval = args.command_func(args, args.config) or 0
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/MishMash-0.3b4-py3.6.egg/mishmash/core.py", line 20, in run
    retval = super().run(args)
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/nicfit/command.py", line 62, in run
    return self._run()
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/MishMash-0.3b4-py3.6.egg/mishmash/commands/sync/sync.py", line 458, in _run
    result = _syncLib(lib)
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/MishMash-0.3b4-py3.6.egg/mishmash/commands/sync/sync.py", line 451, in _syncLib
    return eyed3_main(args, None)
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/eyed3/main.py", line 50, in main
    fs_encoding=args.fs_encoding)
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/eyed3/utils/__init__.py", line 112, in walk
    handler.handleDirectory(root, files)
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/MishMash-0.3b4-py3.6.egg/mishmash/commands/sync/sync.py", line 324, in handleDirectory
    session)
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/MishMash-0.3b4-py3.6.egg/mishmash/commands/sync/sync.py", line 161, in _syncAudioFile
    .filter_by(path=path, lib_id=self._lib.id).one()
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 2814, in one
    ret = self.one_or_none()
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 2784, in one_or_none
    ret = list(self)
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 2855, in __iter__
    return self._execute_and_instances(context)
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 2878, in _execute_and_instances
    result = conn.execute(querycontext.statement, self._params)
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 945, in execute
    return meth(self, multiparams, params)
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/sqlalchemy/sql/elements.py", line 263, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1053, in _execute_clauseelement
    compiled_sql, distilled_params
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1189, in _execute_context
    context)
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1405, in _handle_dbapi_exception
    util.reraise(*exc_info)
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 187, in reraise
    raise value
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1182, in _execute_context
    context)
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 470, in do_execute
    cursor.execute(statement, parameters)
UnicodeEncodeError: 'utf-8' codec can't encode character '\udce9' in position 104: surrogates not allowed
General error:
Traceback (most recent call last):
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/MishMash-0.3b4-py3.6.egg/mishmash/__main__.py", line 53, in main
    retval = args.command_func(args, args.config) or 0
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/MishMash-0.3b4-py3.6.egg/mishmash/core.py", line 20, in run
    retval = super().run(args)
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/nicfit/command.py", line 62, in run
    return self._run()
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/MishMash-0.3b4-py3.6.egg/mishmash/commands/sync/sync.py", line 458, in _run
    result = _syncLib(lib)
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/MishMash-0.3b4-py3.6.egg/mishmash/commands/sync/sync.py", line 451, in _syncLib
    return eyed3_main(args, None)
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/eyed3/main.py", line 50, in main
    fs_encoding=args.fs_encoding)
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/eyed3/utils/__init__.py", line 112, in walk
    handler.handleDirectory(root, files)
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/MishMash-0.3b4-py3.6.egg/mishmash/commands/sync/sync.py", line 324, in handleDirectory
    session)
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/MishMash-0.3b4-py3.6.egg/mishmash/commands/sync/sync.py", line 161, in _syncAudioFile
    .filter_by(path=path, lib_id=self._lib.id).one()
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 2814, in one
    ret = self.one_or_none()
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 2784, in one_or_none
    ret = list(self)
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 2855, in __iter__
    return self._execute_and_instances(context)
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 2878, in _execute_and_instances
    result = conn.execute(querycontext.statement, self._params)
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 945, in execute
    return meth(self, multiparams, params)
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/sqlalchemy/sql/elements.py", line 263, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1053, in _execute_clauseelement
    compiled_sql, distilled_params
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1189, in _execute_context
    context)
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1405, in _handle_dbapi_exception
    util.reraise(*exc_info)
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 187, in reraise
    raise value
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1182, in _execute_context
    context)
  File "/home/baron/src/unsonic/build/venv/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 470, in do_execute
    cursor.execute(statement, parameters)
UnicodeEncodeError: 'utf-8' codec can't encode character '\udce9' in position 104: surrogates not allowed
2017-11-24 09:51:45,044 DEBUG [unsonic][MainThread] Application._atexit
baron@dib ~/src/unsonic $
redshodan commented 6 years ago

It has to do, somehow that isn't immediately obvious, with the path variable in the sync._syncAudioFile() function. audo_file.path is fine, and path is set equal to that but its bad.