search

google-code-export / evennia

Automatically exported from code.google.com/p/evennia
Other
1 stars 0 forks source link

postgreSQL is not working after manage.py migrate #151

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
A) What steps will reproduce the problem?

1. install evennia and also postgresql and psycopg2
2. creating some user and database for evnnia
3. configure the settings.py with something like: 
django.db.backends.postgresql_psycopg2
4. run manage.py syncdb, see how some tables are created on the database
5. after this manage.py migrate for south
5. and see the error message like "django.db.utils.DatabaseError: FEHLER:  
Relation »objects_objectdb« existiert nicht" 

B) What is the expected output? What do you see instead (tracebacks, error
messages)?

my settings.py
>>>SNIP>>>
DATABASES = {
    'default':{
        'ENGINE':'django.db.backends.postgresql_psycopg2',
        'NAME':'evennia',
        'USER':'evennia',
        'PASSWORD':'asdfxyvysdf',
        'HOST':'localhost',
        'PORT':'5432'
        }}

DATABASE_ENGINE = 'postgresql_psycopg2'
DATABASE_NAME = 'evennia'
DATABASE_USER = 'evennia'
DATABASE_PASSWORD = 'asdfxyvysdf'
DATABASE_HOST = 'localhost'
DATABASE_PORT = '5432'
<<<SNAP<<<

"manage.py syncdb" worked (and around ~20 tables are created), but after I 
input "manage.py migrate" i received the following error:

>>>SNIP>>>
  File "/home/lestat/Development/mud/lib/python2.6/site-packages/south/migration/__init__.py", line 191, in migrate_app
    success = migrator.migrate_many(target, workplan, database)
  File "/home/lestat/Development/mud/lib/python2.6/site-packages/south/migration/migrators.py", line 221, in migrate_many
    result = migrator.__class__.migrate_many(migrator, target, migrations, database)
  File "/home/lestat/Development/mud/lib/python2.6/site-packages/south/migration/migrators.py", line 292, in migrate_many
    result = self.migrate(migration, database)
  File "/home/lestat/Development/mud/lib/python2.6/site-packages/south/migration/migrators.py", line 125, in migrate
    result = self.run(migration)
  File "/home/lestat/Development/mud/lib/python2.6/site-packages/south/migration/migrators.py", line 99, in run
    return self.run_migration(migration)
  File "/home/lestat/Development/mud/lib/python2.6/site-packages/south/migration/migrators.py", line 82, in run_migration
    south.db.db.execute_deferred_sql()
  File "/home/lestat/Development/mud/lib/python2.6/site-packages/south/db/generic.py", line 184, in execute_deferred_sql
    self.execute(sql)
  File "/home/lestat/Development/mud/lib/python2.6/site-packages/south/db/generic.py", line 150, in execute
    cursor.execute(sql, params)
  File "/home/lestat/Development/mud/lib/python2.6/site-packages/django/db/backends/postgresql_psycopg2/base.py", line 44, in execute
    return self.cursor.execute(query, args)
django.db.utils.DatabaseError: FEHLER:  Relation »objects_objectdb« existiert 
nicht
<<<SNAP<<<

In my postgresql I see some table "south_migrationhistory" for example but sure 
"evennia.py -i start" don't work:

>>>SNIP>>>
eObj in d, d
  File "/home/lestat/Development/mud/evennia/src/server/server.py", line 183, in <module>
    EVENNIA = Evennia(application)
  File "/home/lestat/Development/mud/evennia/src/server/server.py", line 77, in __init__
    self.run_initial_setup()
  File "/home/lestat/Development/mud/evennia/src/server/server.py", line 131, in run_initial_setup
    initial_setup.handle_setup(0)
  File "/home/lestat/Development/mud/evennia/src/server/initial_setup.py", line 235, in handle_setup
    for obj in ObjectDB.objects.all():
  File "/home/lestat/Development/mud/lib/python2.6/site-packages/django/db/models/query.py", line 105, in _result_iter
    self._fill_cache()
  File "/home/lestat/Development/mud/lib/python2.6/site-packages/django/db/models/query.py", line 783, in _fill_cache
    self._result_cache.append(self._iter.next())
  File "/home/lestat/Development/mud/lib/python2.6/site-packages/django/db/models/query.py", line 271, in iterator
    for row in compiler.results_iter():
  File "/home/lestat/Development/mud/lib/python2.6/site-packages/django/db/models/sql/compiler.py", line 677, in results_iter
    for rows in self.execute_sql(MULTI):
  File "/home/lestat/Development/mud/lib/python2.6/site-packages/django/db/models/sql/compiler.py", line 732, in execute_sql
    cursor.execute(sql, params)
  File "/home/lestat/Development/mud/lib/python2.6/site-packages/django/db/backends/postgresql_psycopg2/base.py", line 44, in execute
    return self.cursor.execute(query, args)
django.db.utils.DatabaseError: FEHLER:  aktuelle Transaktion wurde abgebrochen, 
Befehle werden bis zum Ende der Transaktion ignoriert

Failed to load application: FEHLER:  aktuelle Transaktion wurde abgebrochen, 
Befehle werden bis zum Ende der Transaktion ignoriert
<<<SNIP<<<

If I doing now again "migrate.py syncdb" I receive this output:

>>>SNIP>>>
(mud)lestat@lestat:~/Development/mud/evennia/game$ ./manage.py syncdb
Syncing...
No fixtures found.

Synced:
 > django.contrib.auth
 > django.contrib.sites
 > django.contrib.contenttypes
 > django.contrib.sessions
 > django.contrib.admin
 > django.contrib.admindocs
 > django.contrib.flatpages
 > src.imc2
 > src.irc
 > src.web.news
 > src.web.website
 > south

Not synced (use migrations):
 - src.config
 - src.players
 - src.objects
 - src.comms
 - src.help
 - src.scripts
(use ./manage.py migrate to migrate these)
<<<SNAP<<<

but sure manage.py migrate leads to the error :-(

C) What version/svn revision of Evennia are you using? On what operating
system? Give as much information as possible to make it easier for
developers to recreate the problem.
 * Revision 1394.

D) Please provide any additional information/suggestions below.

PostgreSQL 8.4.7

My Install-Documentation can be found here: 
http://lotek.heavy.ch/evennia#Database

bye
Marcel

Original issue reported on code.google.com by mar...@heavy.ch on 23 Mar 2011 at 9:50

GoogleCodeExporter commented 9 years ago 
It looks like it's trying to apply table migrations related to objects_objects 
for another app, but objects_objects doesn't exist yet. I can try to get this 
fixed for good this weekend, but until then, maybe try something like this:

./manage.py migrate objects
./manage.py migrate

Here's what we'll probably need to do to fix this for good: 
http://south.aeracode.org/docs/dependencies.html#dependencies

Original comment by snagglepants on 24 Mar 2011 at 5:58

GoogleCodeExporter commented 9 years ago 
thanks for your answer, still after migrate objects receive some errors. also 
doing some stuff like migrate players leeds to an error.

for example:

(mud)lestat@lestat:~/Development/mud/evennia/game$ ./manage.py migrate objects
Running migrations for objects:
 - Migrating forwards to 0001_initial.
 > objects:0001_initial
Traceback (most recent call last):
  File "./manage.py", line 148, in <module>
    execute_manager(settings)
  File "/home/lestat/Development/mud/lib/python2.6/site-packages/django/core/management/__init__.py", line 438, in execute_manager
    utility.execute()
  File "/home/lestat/Development/mud/lib/python2.6/site-packages/django/core/management/__init__.py", line 379, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/lestat/Development/mud/lib/python2.6/site-packages/django/core/management/base.py", line 191, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/home/lestat/Development/mud/lib/python2.6/site-packages/django/core/management/base.py", line 220, in execute
    output = self.handle(*args, **options)
  File "/home/lestat/Development/mud/lib/python2.6/site-packages/south/management/commands/migrate.py", line 105, in handle
    ignore_ghosts = ignore_ghosts,
  File "/home/lestat/Development/mud/lib/python2.6/site-packages/south/migration/__init__.py", line 191, in migrate_app
    success = migrator.migrate_many(target, workplan, database)
  File "/home/lestat/Development/mud/lib/python2.6/site-packages/south/migration/migrators.py", line 221, in migrate_many
    result = migrator.__class__.migrate_many(migrator, target, migrations, database)
  File "/home/lestat/Development/mud/lib/python2.6/site-packages/south/migration/migrators.py", line 292, in migrate_many
    result = self.migrate(migration, database)
  File "/home/lestat/Development/mud/lib/python2.6/site-packages/south/migration/migrators.py", line 125, in migrate
    result = self.run(migration)
  File "/home/lestat/Development/mud/lib/python2.6/site-packages/south/migration/migrators.py", line 99, in run
    return self.run_migration(migration)
  File "/home/lestat/Development/mud/lib/python2.6/site-packages/south/migration/migrators.py", line 82, in run_migration
    south.db.db.execute_deferred_sql()
  File "/home/lestat/Development/mud/lib/python2.6/site-packages/south/db/generic.py", line 184, in execute_deferred_sql
    self.execute(sql)
  File "/home/lestat/Development/mud/lib/python2.6/site-packages/south/db/generic.py", line 150, in execute
    cursor.execute(sql, params)
  File "/home/lestat/Development/mud/lib/python2.6/site-packages/django/db/backends/postgresql_psycopg2/base.py", line 44, in execute
    return self.cursor.execute(query, args)
django.db.utils.DatabaseError: FEHLER:  Relation »players_playerdb« existiert 
nicht

but no hurry, im just trying out if it can works with postgresql, thanks!

bye
Marcel

Original comment by mar...@heavy.ch on 24 Mar 2011 at 8:20

GoogleCodeExporter commented 9 years ago 
This should be resolved as of r1397. Please give it a shot and re-open this if 
it still breaks for you.

Original comment by snagglepants on 24 Mar 2011 at 9:07

GoogleCodeExporter commented 9 years ago 
I dropped my database, recreated and started again. now migrate works flawless, 
but starting evennia.py -i start leads to another error "Failed to load 
application: can't adapt type 'User'" see down my logs..

    application = service.loadApplication(filename, style, passphrase)
  File "/home/lestat/Development/mud/lib/python2.6/site-packages/twisted/application/service.py", line 400, in loadApplication
    application = sob.loadValueFromFile(filename, 'application', passphrase)
  File "/home/lestat/Development/mud/lib/python2.6/site-packages/twisted/persisted/sob.py", line 210, in loadValueFromFile
    exec fileObj in d, d
  File "/home/lestat/Development/mud/evennia/src/server/server.py", line 183, in <module>
    EVENNIA = Evennia(application)
  File "/home/lestat/Development/mud/evennia/src/server/server.py", line 77, in __init__
    self.run_initial_setup()
  File "/home/lestat/Development/mud/evennia/src/server/server.py", line 131, in run_initial_setup
    initial_setup.handle_setup(0)
  File "/home/lestat/Development/mud/evennia/src/server/initial_setup.py", line 229, in handle_setup
    setup_func()
  File "/home/lestat/Development/mud/evennia/src/server/initial_setup.py", line 116, in create_channels
    ChannelConnection.objects.create_connection(goduser, pchan)
  File "/home/lestat/Development/mud/evennia/src/comms/managers.py", line 302, in create_connection
    player = to_object(player)
  File "/home/lestat/Development/mud/evennia/src/comms/managers.py", line 31, in to_object
    if umatch:
  File "/home/lestat/Development/mud/lib/python2.6/site-packages/django/db/models/query.py", line 111, in __nonzero__
    iter(self).next()
  File "/home/lestat/Development/mud/lib/python2.6/site-packages/django/db/models/query.py", line 105, in _result_iter
    self._fill_cache()
  File "/home/lestat/Development/mud/lib/python2.6/site-packages/django/db/models/query.py", line 783, in _fill_cache
    self._result_cache.append(self._iter.next())
  File "/home/lestat/Development/mud/lib/python2.6/site-packages/django/db/models/query.py", line 271, in iterator
    for row in compiler.results_iter():
  File "/home/lestat/Development/mud/lib/python2.6/site-packages/django/db/models/sql/compiler.py", line 677, in results_iter
    for rows in self.execute_sql(MULTI):
  File "/home/lestat/Development/mud/lib/python2.6/site-packages/django/db/models/sql/compiler.py", line 732, in execute_sql
    cursor.execute(sql, params)
  File "/home/lestat/Development/mud/lib/python2.6/site-packages/django/db/backends/postgresql_psycopg2/base.py", line 44, in execute
    return self.cursor.execute(query, args)
django.db.utils.DatabaseError: can't adapt type 'User'

Failed to load application: can't adapt type 'User'

Original comment by mar...@heavy.ch on 24 Mar 2011 at 9:21

GoogleCodeExporter commented 9 years ago

Original comment by griatch on 24 Mar 2011 at 10:51

GoogleCodeExporter commented 9 years ago 
I cannot test this properly, but since it doesn't seem resolved, I'm labeling 
it "Accepted" again.
.
Griatch

Original comment by griatch on 15 Apr 2011 at 11:52

GoogleCodeExporter commented 9 years ago 
[deleted comment]
GoogleCodeExporter commented 9 years ago

Original comment by griatch on 25 Apr 2011 at 9:09

GoogleCodeExporter commented 9 years ago 
Is this still an issue? Could someone with postgreSQL check?

Original comment by griatch on 28 May 2011 at 7:43

GoogleCodeExporter commented 9 years ago 
For me is still not working (with the latest svn update, fresh database, 
etcetera).

1. syncdb works (as before)
2. migrate not (manage.py migrate leads to an "database error")
3. running without migrate, just evennia.py -i start leads to an 
"django.db.utils.DatabaseError: FEHLER:  Relation »server_serverconfig« 
existiert nicht LINE 1: ..."."db_key", "server_serverconfig"."db_value" FROM 
"server_se..."

***************
Information:
***************
 * OS: Ubuntu 10.04 (64bit)
 * Evennia: Revision 1540.
 * Python: 2.6.5
 * Django-1.2.5 / Twisted-10.2.0
 * South-0.7.3
 * psycopg2-2.4
 * zope.interface-3.6.1

***************
settings.py:
***************
>>>snip>>>
DATABASES = {
    'default':{
        'ENGINE':'django.db.backends.postgresql_psycopg2',
        'NAME':'evennia',
        'USER':'evennia',
        'PASSWORD':'xxxxxxxxxxxxx',
        'HOST':'localhost',
        'PORT':'5432'
        }}

DATABASE_ENGINE = 'postgresql_psycopg2'
DATABASE_NAME = 'evennia'
DATABASE_USER = 'evennia'
DATABASE_PASSWORD = 'xxxxxxxxxxxxx'
DATABASE_HOST = 'localhost'
DATABASE_PORT = '5432'
<<<snap<<<

***************
INPUT MIGRATE
***************
(mud)lestat@lestat:~/Development/mud/evennia/game$ ./manage.py migrate

***************
OUTPUT
***************
Running migrations for server:
 - Migrating forwards to 0002_fix_config_value.
 > server:0001_rename_config_table_to_server_table
Traceback (most recent call last):
  File "./manage.py", line 143, in <module>
    execute_manager(settings)
  File "/home/lestat/Development/mud/lib/python2.6/site-packages/django/core/management/__init__.py", line 438, in execute_manager
    utility.execute()
  File "/home/lestat/Development/mud/lib/python2.6/site-packages/django/core/management/__init__.py", line 379, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/lestat/Development/mud/lib/python2.6/site-packages/django/core/management/base.py", line 191, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/home/lestat/Development/mud/lib/python2.6/site-packages/django/core/management/base.py", line 220, in execute
    output = self.handle(*args, **options)
  File "/home/lestat/Development/mud/lib/python2.6/site-packages/south/management/commands/migrate.py", line 105, in handle
    ignore_ghosts = ignore_ghosts,
  File "/home/lestat/Development/mud/lib/python2.6/site-packages/south/migration/__init__.py", line 191, in migrate_app
    success = migrator.migrate_many(target, workplan, database)
  File "/home/lestat/Development/mud/lib/python2.6/site-packages/south/migration/migrators.py", line 221, in migrate_many
    result = migrator.__class__.migrate_many(migrator, target, migrations, database)
  File "/home/lestat/Development/mud/lib/python2.6/site-packages/south/migration/migrators.py", line 292, in migrate_many
    result = self.migrate(migration, database)
  File "/home/lestat/Development/mud/lib/python2.6/site-packages/south/migration/migrators.py", line 125, in migrate
    result = self.run(migration)
  File "/home/lestat/Development/mud/lib/python2.6/site-packages/south/migration/migrators.py", line 99, in run
    return self.run_migration(migration)
  File "/home/lestat/Development/mud/lib/python2.6/site-packages/south/migration/migrators.py", line 81, in run_migration
    migration_function()
  File "/home/lestat/Development/mud/lib/python2.6/site-packages/south/migration/migrators.py", line 57, in <lambda>
    return (lambda: direction(orm))
  File "/home/lestat/Development/mud/evennia/src/server/migrations/0001_rename_config_table_to_server_table.py", line 24, in forwards
    ('db_value', self.gf('django.db.models.fields.TextField')(blank=True)),
  File "/home/lestat/Development/mud/lib/python2.6/site-packages/south/db/generic.py", line 226, in create_table
    ', '.join([col for col in columns if col]),
  File "/home/lestat/Development/mud/lib/python2.6/site-packages/south/db/generic.py", line 150, in execute
    cursor.execute(sql, params)
  File "/home/lestat/Development/mud/lib/python2.6/site-packages/django/db/backends/postgresql_psycopg2/base.py", line 44, in execute
    return self.cursor.execute(query, args)
django.db.utils.DatabaseError: FEHLER:  aktuelle Transaktion wurde abgebrochen, 
Befehle werden bis zum Ende der Transaktion ignoriert

Original comment by mar...@heavy.ch on 28 May 2011 at 9:39

GoogleCodeExporter commented 9 years ago 
I am getting the same or similar errors with both "migrate" and "migrate 
objects".

With "migrate" the error results in:

django.db.utils.DatabaseError: current transaction is aborted, commands ignored 
until end of transaction block

This seems to be caused by a malformed query being passed to south...I think.
Which ultimately causes the execution in 
django/db/backends/postgresql_psycopg2/base.py to fail...I think.
My debugger suggests and internal error state at line 44 of base.py, but being 
a noob I'm not sure what this means...I think. ;)

With "migrate objects", as suggested up the page, results in the following:

 - Migrating forwards to 0006_add_view_lock.
 > objects:0006_add_view_lock
Traceback (most recent call last):
  File "./manage.py", line 143, in <module>

...

  File "/home/morn/morn/src/objects/migrations/0006_add_view_lock.py", line 23, in forwards
    except utils.DatabaseError:
NameError: global name 'utils' is not defined

It seems for some reason the utils.DatabaseError method is not being imported 
properly.

I confirmed this by adding:

from django.db import utils

at the top of src/objects/migrations/0006_add_view_lock.py.

The command "./migrate.py migrate objects" was then able to complete 
successfully.

Suggest changing line 5 of src/objects/migrations/0006_add_view_lock.py
from "from django.db import models"
to "from django.db import models, utils".

Hope this helps someone or leads to further information.

Original comment by chris.ga...@gmail.com on 29 Jun 2011 at 3:22

GoogleCodeExporter commented 9 years ago 
@Chris

Yes, as you note, it's just to add the proper import statement. The problem is 
unrelated to the original issue reported here (which has to do with differences 
between database systems). The missing import is now fixed in r1566, thanks for 
the heads up.
.
Griatch

Original comment by griatch on 29 Jun 2011 at 6:02

GoogleCodeExporter commented 9 years ago 
I must admit I'm quite baffled by this one. Having set up postgresql on my 
development system, I can reproduce this error, and it makes no sense. It 
crashes on a migration where a table (ServerConfig) is being created. It quits 
upon executing the sql, but if I run the exact same sql string from the 
postgresql prompt it works fine! Even forcing the sql directly to South chokes 
it up. 
As far as I can tell it looks the same as other migrations doing similar thing, 
but that works fine. Most confusing. 

As a temporary workaround one can turn off migrations for postgresql by adding 
the line 
 `INSTALLED_APPS = INSTALLED_APPS[:-2]`
to the game/settings.py file. Doing a plain syncdb works fine, then I get 
errors during initial_startup ... something fishy is going on with postgresql, 
that's for sure. :-( 
.
Griatch

Original comment by griatch on 13 Nov 2011 at 10:01

GoogleCodeExporter commented 9 years ago 
Update, June 2012: With the latest update, the migration errors under 
postgresql seems to be gone. It however still fails on first server start, when 
trying to create the default channels. The error is different: 

File 
"/home/griatch/Devel/MUD/evennia/evennia-trunk/src/server/initial_setup.py", 
line 106, in create_channels
    PlayerChannelConnection.objects.create_connection(goduser, pchan)
  File "/home/griatch/Devel/MUD/evennia/evennia-trunk/src/comms/managers.py", line 362, in create_connection
    player = to_object(player)
  File "/home/griatch/Devel/MUD/evennia/evennia-trunk/src/comms/managers.py", line 32, in to_object
    if umatch:
  File "/usr/lib/python2.7/dist-packages/django/db/models/query.py", line 130, in __nonzero__
    iter(self).next()
  File "/usr/lib/python2.7/dist-packages/django/db/models/query.py", line 118, in _result_iter
    self._fill_cache()
  File "/usr/lib/python2.7/dist-packages/django/db/models/query.py", line 875, in _fill_cache
    self._result_cache.append(self._iter.next())
  File "/usr/lib/python2.7/dist-packages/django/db/models/query.py", line 291, in iterator
    for row in compiler.results_iter():
  File "/usr/lib/python2.7/dist-packages/django/db/models/sql/compiler.py", line 763, in results_iter
    for rows in self.execute_sql(MULTI):
  File "/usr/lib/python2.7/dist-packages/django/db/models/sql/compiler.py", line 818, in execute_sql
    cursor.execute(sql, params)
  File "/usr/lib/python2.7/dist-packages/django/db/backends/postgresql_psycopg2/base.py", line 52, in execute
    return self.cursor.execute(query, args)
django.db.utils.DatabaseError: can't adapt type 'User'

This seems to be related to a type error, but this would happen in postgresql 
and not in sqlite3 nor in mysql is unclear to me.

Original comment by griatch on 14 Jun 2012 at 8:23

GoogleCodeExporter commented 9 years ago 
Ugh. The problem is related to the funny way psycopg2 converts things. It's in 
the FAQ:

http://packages.python.org/psycopg2/faq.html#problems-with-type-conversions

It's not an issue in sqlite3 and mysql because their backends are more 
aggressive about converting types. psycopg2, though, will throw this cryptic 
can't adapt error when it encounters a type it doesn't know how to handle. In 
this case, the type is django.contrib.auth.models.User

My understanding of the codebase is still fresh, but I bet that type either 
needs a getquoted() method, a register_adapter for the type needs to be added 
for it in the psycopg2 base.py (I tried this briefly, but couldn't get the 
import to work), or evennia would have to convert to something else (like a 
string) before passing it to Django.

Ugh indeed.

Original comment by Zeta142...@gmail.com on 16 Jun 2012 at 8:51

GoogleCodeExporter commented 9 years ago 
@Zeta142125: 
Thanks for the pointer. Unfortunately, the User type is not a part of Evennia, 
but is a standard Django component. It's possible that the issue lies in some 
of the connections the User does to the Player object, but it is proving hard 
to test since these same commands triggering the traceback works just fine when 
testing them a python shell. 

The error appears during the linking of Player/User #1 to the default channels. 
In the latest push (r0280adbf6a05), I have disabled this linking whenever the 
server detects postgresql-psycopg2. It instead only creates the channels and 
displays a warning that one needs to do this linkage manually in-game (this 
works normally, strangely enough). 

It also seems that postgres-psycopg2 needs to use the 
OPTION={"autocommit":True} in the DATABASE dictionary settings file. I saw a 
bunch of weird errors during mass-deletion of objects without it. Will add that 
to the wiki. 

Postgresql-psycopg2 works with this fix, but since it's unclear what the actual 
problem is, it's hardly satisfactory (not to mention that it might hide further 
problems not as easily detected). I'm closing this Issue and opening a less 
critical Issue instead.

Original comment by griatch on 16 Jun 2012 at 1:03

GoogleCodeExporter commented 9 years ago 
The new Issue that should be used for updates to this strange behaviour is 
Issue 241.
.
Griatch

Original comment by griatch on 17 Jun 2012 at 8:16