fastmonkeys / stellar

Fast database snapshot and restore tool for development
MIT License
3.86k stars 119 forks source link

Unicode-objects must be encoded before hashing #39

Closed synotna closed 9 years ago

synotna commented 10 years ago

Tested on both an empty and new django database

$ stellar snapshot
Traceback (most recent call last):
  File "/home/antony/.pyenv/versions/et/lib/python3.4/site-packages/sqlalchemy/engine/base.py", line 889, in _execute_context
    context = constructor(dialect, self, conn, *args)
  File "/home/antony/.pyenv/versions/et/lib/python3.4/site-packages/sqlalchemy/engine/default.py", line 560, in _init_compiled
    self.__process_defaults()
  File "/home/antony/.pyenv/versions/et/lib/python3.4/site-packages/sqlalchemy/engine/default.py", line 942, in __process_defaults
    val = self.get_insert_default(c)
  File "/home/antony/.pyenv/versions/et/lib/python3.4/site-packages/sqlalchemy/dialects/postgresql/base.py", line 1632, in get_insert_default
    return super(PGExecutionContext, self).get_insert_default(column)
  File "/home/antony/.pyenv/versions/et/lib/python3.4/site-packages/sqlalchemy/engine/default.py", line 896, in get_insert_default
    return self._exec_default(column.default, column.type)
  File "/home/antony/.pyenv/versions/et/lib/python3.4/site-packages/sqlalchemy/engine/default.py", line 882, in _exec_default
    return default.arg(self)
  File "/home/antony/.pyenv/versions/et/lib/python3.4/site-packages/sqlalchemy/sql/schema.py", line 1926, in <lambda>
    return lambda ctx: fn()
  File "/home/antony/.pyenv/versions/et/lib/python3.4/site-packages/stellar/models.py", line 12, in get_unique_hash
    return hashlib.md5(str(uuid.uuid4())).hexdigest()
TypeError: Unicode-objects must be encoded before hashing

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/antony/.pyenv/versions/et/bin/stellar", line 9, in <module>
    load_entry_point('stellar==0.4.1', 'console_scripts', 'stellar')()
  File "/home/antony/.pyenv/versions/et/lib/python3.4/site-packages/stellar/command.py", line 264, in main
    stellar()
  File "/home/antony/.pyenv/versions/et/lib/python3.4/site-packages/click/core.py", line 610, in __call__
    return self.main(*args, **kwargs)
  File "/home/antony/.pyenv/versions/et/lib/python3.4/site-packages/click/core.py", line 590, in main
    rv = self.invoke(ctx)
  File "/home/antony/.pyenv/versions/et/lib/python3.4/site-packages/click/core.py", line 936, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/antony/.pyenv/versions/et/lib/python3.4/site-packages/click/core.py", line 782, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/antony/.pyenv/versions/et/lib/python3.4/site-packages/click/core.py", line 416, in invoke
    return callback(*args, **kwargs)
  File "/home/antony/.pyenv/versions/et/lib/python3.4/site-packages/stellar/command.py", line 70, in snapshot
    app.create_snapshot(name, before_copy=before_copy)
  File "/home/antony/.pyenv/versions/et/lib/python3.4/site-packages/stellar/app.py", line 105, in create_snapshot
    self.db.session.flush()
  File "/home/antony/.pyenv/versions/et/lib/python3.4/site-packages/sqlalchemy/orm/session.py", line 1919, in flush
    self._flush(objects)
  File "/home/antony/.pyenv/versions/et/lib/python3.4/site-packages/sqlalchemy/orm/session.py", line 2037, in _flush
    transaction.rollback(_capture_exception=True)
  File "/home/antony/.pyenv/versions/et/lib/python3.4/site-packages/sqlalchemy/util/langhelpers.py", line 60, in __exit__
    compat.reraise(exc_type, exc_value, exc_tb)
  File "/home/antony/.pyenv/versions/et/lib/python3.4/site-packages/sqlalchemy/util/compat.py", line 182, in reraise
    raise value
  File "/home/antony/.pyenv/versions/et/lib/python3.4/site-packages/sqlalchemy/orm/session.py", line 2001, in _flush
    flush_context.execute()
  File "/home/antony/.pyenv/versions/et/lib/python3.4/site-packages/sqlalchemy/orm/unitofwork.py", line 372, in execute
    rec.execute(self)
  File "/home/antony/.pyenv/versions/et/lib/python3.4/site-packages/sqlalchemy/orm/unitofwork.py", line 526, in execute
    uow
  File "/home/antony/.pyenv/versions/et/lib/python3.4/site-packages/sqlalchemy/orm/persistence.py", line 65, in save_obj
    mapper, table, insert)
  File "/home/antony/.pyenv/versions/et/lib/python3.4/site-packages/sqlalchemy/orm/persistence.py", line 602, in _emit_insert_statements
    execute(statement, params)
  File "/home/antony/.pyenv/versions/et/lib/python3.4/site-packages/sqlalchemy/engine/base.py", line 729, in execute
    return meth(self, multiparams, params)
  File "/home/antony/.pyenv/versions/et/lib/python3.4/site-packages/sqlalchemy/sql/elements.py", line 322, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/home/antony/.pyenv/versions/et/lib/python3.4/site-packages/sqlalchemy/engine/base.py", line 826, in _execute_clauseelement
    compiled_sql, distilled_params
  File "/home/antony/.pyenv/versions/et/lib/python3.4/site-packages/sqlalchemy/engine/base.py", line 893, in _execute_context
    None, None)
  File "/home/antony/.pyenv/versions/et/lib/python3.4/site-packages/sqlalchemy/engine/base.py", line 1159, in _handle_dbapi_exception
    exc_info
  File "/home/antony/.pyenv/versions/et/lib/python3.4/site-packages/sqlalchemy/util/compat.py", line 188, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=exc_value)
  File "/home/antony/.pyenv/versions/et/lib/python3.4/site-packages/sqlalchemy/util/compat.py", line 181, in reraise
    raise value.with_traceback(tb)
  File "/home/antony/.pyenv/versions/et/lib/python3.4/site-packages/sqlalchemy/engine/base.py", line 889, in _execute_context
    context = constructor(dialect, self, conn, *args)
  File "/home/antony/.pyenv/versions/et/lib/python3.4/site-packages/sqlalchemy/engine/default.py", line 560, in _init_compiled
    self.__process_defaults()
  File "/home/antony/.pyenv/versions/et/lib/python3.4/site-packages/sqlalchemy/engine/default.py", line 942, in __process_defaults
    val = self.get_insert_default(c)
  File "/home/antony/.pyenv/versions/et/lib/python3.4/site-packages/sqlalchemy/dialects/postgresql/base.py", line 1632, in get_insert_default
    return super(PGExecutionContext, self).get_insert_default(column)
  File "/home/antony/.pyenv/versions/et/lib/python3.4/site-packages/sqlalchemy/engine/default.py", line 896, in get_insert_default
    return self._exec_default(column.default, column.type)
  File "/home/antony/.pyenv/versions/et/lib/python3.4/site-packages/sqlalchemy/engine/default.py", line 882, in _exec_default
    return default.arg(self)
  File "/home/antony/.pyenv/versions/et/lib/python3.4/site-packages/sqlalchemy/sql/schema.py", line 1926, in <lambda>
    return lambda ctx: fn()
  File "/home/antony/.pyenv/versions/et/lib/python3.4/site-packages/stellar/models.py", line 12, in get_unique_hash
    return hashlib.md5(str(uuid.uuid4())).hexdigest()
sqlalchemy.exc.StatementError: Unicode-objects must be encoded before hashing (original cause: TypeError: Unicode-objects must be encoded before hashing) "INSERT INTO snapshot (id, snapshot_name, project_name, hash, created_at, worker_p$1
synotna commented 9 years ago
antony@synotna:~/compete/code$ stellar snapshot
Snapshotting database compete-dev
Traceback (most recent call last):
  File "/home/antony/.pyenv/versions/compete/bin/stellar", line 9, in <module>
    load_entry_point('stellar==0.4.2', 'console_scripts', 'stellar')()
  File "/home/antony/.pyenv/versions/compete/lib/python3.4/site-packages/stellar/command.py", line 264, in main
    stellar()
  File "/home/antony/.pyenv/versions/compete/lib/python3.4/site-packages/click/core.py", line 572, in __call__
    return self.main(*args, **kwargs)
  File "/home/antony/.pyenv/versions/compete/lib/python3.4/site-packages/click/core.py", line 552, in main
    rv = self.invoke(ctx)
  File "/home/antony/.pyenv/versions/compete/lib/python3.4/site-packages/click/core.py", line 893, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/antony/.pyenv/versions/compete/lib/python3.4/site-packages/click/core.py", line 744, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/antony/.pyenv/versions/compete/lib/python3.4/site-packages/click/core.py", line 388, in invoke
    return callback(*args, **kwargs)
  File "/home/antony/.pyenv/versions/compete/lib/python3.4/site-packages/stellar/command.py", line 70, in snapshot
    app.create_snapshot(name, before_copy=before_copy)
  File "/home/antony/.pyenv/versions/compete/lib/python3.4/site-packages/stellar/app.py", line 116, in create_snapshot
    table.get_table_name('master')
  File "/home/antony/.pyenv/versions/compete/lib/python3.4/site-packages/stellar/models.py", line 66, in get_table_name
    ).hexdigest()[0:16]
TypeError: Unicode-objects must be encoded before hashing
Teemu commented 9 years ago

Oh boy.

ptype commented 9 years ago

fyi - seeing same issue on a non-Django db. Trace seems similar to above, just after running init:

p: ~ $ stellar snapshot pjan16
Snapshotting database db_xx
Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.4/bin/stellar", line 9, in <module>
    load_entry_point('stellar==0.4.2', 'console_scripts', 'stellar')()
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/stellar/command.py", line 264, in main
    stellar()
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/click/core.py", line 572, in __call__
    return self.main(*args, **kwargs)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/click/core.py", line 552, in main
    rv = self.invoke(ctx)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/click/core.py", line 893, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/click/core.py", line 744, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/click/core.py", line 388, in invoke
    return callback(*args, **kwargs)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/stellar/command.py", line 70, in snapshot
    app.create_snapshot(name, before_copy=before_copy)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/stellar/app.py", line 116, in create_snapshot
    table.get_table_name('master')
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/stellar/models.py", line 66, in get_table_name
    ).hexdigest()[0:16]
TypeError: Unicode-objects must be encoded before hashing
vendion commented 9 years ago

I'm seeing this as well, using Postgres 9.3 as the database I'm connecting to

╭─vendion@tyre ~/Projects/vendionSite  ‹develop*› 
╰─➤  stellar snapshot
Snapshotting database vendionSite
Traceback (most recent call last):
  File "/usr/bin/stellar", line 9, in <module>
    load_entry_point('stellar==0.4.2', 'console_scripts', 'stellar')()
  File "/usr/lib/python3.4/site-packages/stellar/command.py", line 264, in main
    stellar()
  File "/usr/lib/python3.4/site-packages/click/core.py", line 572, in __call__
    return self.main(*args, **kwargs)
  File "/usr/lib/python3.4/site-packages/click/core.py", line 552, in main
    rv = self.invoke(ctx)
  File "/usr/lib/python3.4/site-packages/click/core.py", line 893, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/lib/python3.4/site-packages/click/core.py", line 744, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/lib/python3.4/site-packages/click/core.py", line 388, in invoke
    return callback(*args, **kwargs)
  File "/usr/lib/python3.4/site-packages/stellar/command.py", line 70, in snapshot
    app.create_snapshot(name, before_copy=before_copy)
  File "/usr/lib/python3.4/site-packages/stellar/app.py", line 116, in create_snapshot
    table.get_table_name('master')
  File "/usr/lib/python3.4/site-packages/stellar/models.py", line 66, in get_table_name
    ).hexdigest()[0:16]
TypeError: Unicode-objects must be encoded before hashing