Batch Command Processing Issue not present when entered manually at command line

[GoogleCodeExporter]

A) What steps will reproduce the problem?

1. @batchcommand examples.error_test
2. error_test.ev attached

B) What is the expected output? What do you see instead?

Expected same results as command line entry of commands found in error_test.ev

@batchcommand examples.error_test
Running Batch-command processor - Automatic mode for examples.error_test ...
01/04: @dig 5122                                      
Created room '51227' of type game.gamesrc.objects.baseobjects.Room.
02/04: teleport 5122                                  
51227
03/04: @describe 51227 =  Silver like stone has been carefully chiseled and 
embedded into the roadway. Each stone has been polished to a bright hue and 
then 
encrusted with sparkling gems. The gems seem to glow and glitter with an inner 
light.A large brass sign as been bolted atop of a building’s entranceway off 
to 
the south. Large barrels of grain, rice, and fruit have been set just outside 
the doorway. A few clay pots, containing a single rose colored flower tipped 
with light blue, has been lined against the northern wall. A strong scent of 
honey and nectar lingers here as a checkered road beginsoff to the east[...]
Traceback (most recent call last):
  File "C:\Users\John\Mud\evennia\trunk\src\commands\cmdhandler.py", line 376, 
in cmdhandler
    cmd.func()
  File "C:\Users\John\Mud\evennia\trunk\game\gamesrc\commands\default\objmanip.
py", line 1155, in func
    obj.db.desc = desc
  File "C:\Users\John\Mud\evennia\trunk\src\typeclasses\models.py", line 878, in 
__setattr__
    obj.set_attribute(attrname, value)
  File "C:\Users\John\Mud\evennia\trunk\src\typeclasses\models.py", line 769, in 
set_attribute
    new_attrib.value = new_value
  File "C:\Users\John\Mud\evennia\trunk\src\typeclasses\models.py", line 252, in 
value_set
    self.save()
  File "C:\Users\John\Mud\evennia\trunk\src\utils\idmapper\base.py", line 110, 
in save
    super(SharedMemoryModel, self).save(*args, **kwargs)
  File "C:\Python27\lib\site-packages\django\db\models\base.py", line 435, in 
save
    self.save_base(using=using, force_insert=force_insert, 
force_update=force_update)
  File "C:\Python27\lib\site-packages\django\db\models\base.py", line 528, in 
save_base
    result = manager._insert(values, return_id=update_pk, using=using)
  File "C:\Python27\lib\site-packages\django\db\models\manager.py", line 195, in 
_insert
    return insert_query(self.model, values, **kwargs)
  File "C:\Python27\lib\site-packages\django\db\models\query.py", line 1479, in 
insert_query
    return query.get_compiler(using=using).execute_sql(return_id)
  File "C:\Python27\lib\site-packages\django\db\models\sql\compiler.py", line 
783, in execute_sql
    cursor = super(SQLInsertCompiler, self).execute_sql(None)
  File "C:\Python27\lib\site-packages\django\db\models\sql\compiler.py", line 
727, in execute_sql
    cursor.execute(sql, params)
  File "C:\Python27\lib\site-packages\django\db\backends\util.py", line 15, in 
execute
    return self.cursor.execute(sql, params)
  File "C:\Python27\lib\site-packages\django\db\backends\sqlite3\base.py", line 
200, in execute
    return Database.Cursor.execute(self, query, params)
DatabaseError: You must not use 8-bit bytestrings unless you use a text_factory 
that can interpret 8-bit bytestrings (like text_factory = str). It is highly 
recommended that you instead just switch your application to Unicode strings.

Above traceback is from an untrapped error. Please file a bug report.
04/04: @name 51227 = On Sanctuary Place;5122          
Object's name changed to 'On Sanctuary Place' 51227.
End of batch file.
  Batchfile 'examples.error_test' applied.

C) What version/revision of Evennia are you using? On what operating
system? Give as much information as possible to make it easier for
developers to recreate the problem.

Evennia SVN-Alpha R1033
Windows7
Python2.7
Twisted-10.1.0
PIL 1.1.7
Django-1.2.1

D) Please provide any additional information below:

The same commands from the user command line do not fail.

Original issue reported on code.google.com by john.edw...@gmail.com on 3 Sep 2010 at 2:14

Attachments:

• error_test.ev

[GoogleCodeExporter]

The reason for the error is because you have a strange symbol in the text, it's 
the apostrophe in "building’s" in the second line (my browser can't display it 
either); it's likely a special non-ascii symbol on your keyboard. 
That said, Evennia (and Django) supports non-ascii characters by internally 
using the full unicode set, so this should not cause a traceback ... It is 
possible that the batch-process commands do not pass through the same filters 
as text entered on the command line does. I'll look into that, thanks!
.
Griatch

Original comment by griatch on 3 Sep 2010 at 2:25

• Changed state: Accepted

[GoogleCodeExporter]

This is handled in r1038, although it turns out it's probably impossible to 
actually fully resolve. The error comes from your batchfile containing a 
non-ascii character while not being saved in the UTF-8 encoding that Evennia 
(and Python) needs. The telnet protocol is a known encoding to us, which is why 
it works for you on the console, but not in the batchprocessor reading an 
arbitrary file. It's unfortunately a very difficult problem to determine which 
encoding a file is in just by looking at it, so we have to assume one. UTF-8 is 
the Python- and international standard.
If you save the batchfile in UTF-8 it is read and parsed just fine - how to do 
this is different for every text editor however. 

r1038 adds a more useful error message on how to resolve this, rather just 
dumping a traceback. I'll also add a section on it to the documentation. 

Original comment by griatch on 4 Sep 2010 at 8:06

• Changed state: Fixed
• Added labels: Type-Defect