search

lanto03 / couchdb-python

Automatically exported from code.google.com/p/couchdb-python
Other
0 stars 0 forks source link

Cannot add DictField or ListField to existing document schemas #88

Open GoogleCodeExporter opened 8 years ago

GoogleCodeExporter commented 8 years ago 
If a document is loaded, and the ListField/DictField specified in the schema 
class is not pre-
existing, the Fields do not work.

What steps will reproduce the problem?
1. Create and save a document
2. Add ListField/DictField to schema class and re-load the document
3. ListField/DictField are non-functional

What is the expected output? What do you see instead?
The ListField should be represented by a schema.Proxy object, instead it is a 
standard python list 
object. When storing the document, any values placed in the list will be 
ignored.

What version of the product are you using? On what operating system?
Python 2.5.2
CouchDB 0.9.1

Please provide any additional information below.
Small app to demonstrate:
"""
#!/usr/bin/env python

import couchdb
from couchdb import schema

server = couchdb.Server('http://localhost:5984/')
db = server['list-tests']

class TestDoc(schema.Document):
    title = schema.TextField()

class TestDocAlt(schema.Document):
    title = schema.TextField()
    test_list = schema.ListField(schema.TextField())

t1 = TestDoc()
t1.title = 'Test'
t1.store(db)

t2 = TestDocAlt.load(db, t1.id)
# Will print "<type 'list'>" not "<class 'couchdb.schema.Proxy'>"
print type(t2.test_list)

t2.test_list.append('ham')
t2.store(db)

t3 = db.get(t2.id)
# Will print "False" becuase the list was never stored as part of t2 on line 25
print t3.has_key('test_list')
"""

Original issue reported on code.google.com by thret...@gmail.com on 27 Aug 2009 at 6:00

GoogleCodeExporter commented 8 years ago 
I just wanted to report this as well. In the meantime something like this 
(before working with test_list) has 
worked for me:

if "test_list" not in t2: #Enhance document if it doesn't already have test_list
    t2["test_list"] = []

A proceeding store will save the list then.

Original comment by st.schus...@gmail.com on 8 Sep 2009 at 12:30

GoogleCodeExporter commented 8 years ago 
I've actually had issues with this in the past and I think in general it's a 
good problem to solve.  Especially with a 
doc-based DB like couch, the document structure could change all the time.  My 
solution was to write a class 
function in my document classes to do update to the new schema through a 
recursion through the fields in the 
document.  This should deal fine with addition and subtraction of fields, but 
it won't handle name changes since 
it populates fields with the same name.    

In the above example, the way this would be used (if both TestDoc and 
TestDocAlt derive from MGDocument)

t1 = TestDoc()
t1.title = 'Test'
t1.store(db)

t2 = TestDocAlt.load(db, t1.id)
t2 = TestDocAlt.update_schema(t2)
print type(t2.test_list)

and then a 

t2.store(db) 

to persist back to the db.  

class MGDocument(schema.Document):
    @classmethod
    def update_schema(cls, old_doc):
        """
        Run this on an old document to generate the correct 
        schema and update to the correct rev and ids.
        """
        new_doc = cls()
        def update_field(field, old_field):
            try:
                for sub_field in field:
                    if sub_field in old_field:
                        try:
                            update_field(field[sub_field], old_field[sub_field])
                        except TypeError:
                            # Means this field is non-iterable, set with the old value
                            field[sub_field] = old_field[sub_field]
            except TypeError:
                # this field is non-iterable, return to the calling function 
                raise

        update_field(new_doc, old_doc)              

        if '_rev' in old_doc:
            new_doc['_rev'] = old_doc['_rev']                   
        new_doc._set_id(old_doc._get_id())                      
        return new_doc

Original comment by mmar...@gmail.com on 11 Mar 2010 at 7:11

GoogleCodeExporter commented 8 years ago 
Almost three years for changing:
--- a/couchdb/mapping.py
+++ b/couchdb/mapping.py
@@ -175,7 +175,7 @@
     @classmethod
     def wrap(cls, data):
-        instance = cls()
-        instance._data = data
+        instance = cls(**data)
         return instance

     def _to_python(self, value):

However, this patch compromises wrap class method - why it should be exists 
then?(:

Original comment by kxepal on 21 May 2011 at 4:19

GoogleCodeExporter commented 8 years ago 
Added patch with properly handle of private members and tests.

Original comment by kxepal on 21 May 2011 at 4:43

Attachments:

GoogleCodeExporter commented 8 years ago 
I tried the patch from kxepal with the current trunk version. When i run the 
tests against it I get some errors (errors are attached at the end of this 
comment). I solved the problem in a different way. I'm not sure if this is the 
right way but at least it succeeds all test conditions on my system (OS X, 
Python Python 2.6.1 and simplejson installed). Patch for my fix is attached. 
Any comments/improvements appreciated.

python __init__.py
................................................................................
...................F....F................................................
======================================================================
FAIL: Doctest: couchdb.mapping
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/doctest.py", line 2131, in runTest
    raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for couchdb.mapping
  File "/Users/christian/devel/carrot/couchdb-python-vanilla/couchdb/mapping.py", line 8, in mapping

----------------------------------------------------------------------
File "/Users/christian/devel/carrot/couchdb-python-vanilla/couchdb/mapping.py", 
line 32, in couchdb.mapping
Failed example:
    person = Person.load(db, person.id)
Exception raised:
    Traceback (most recent call last):
      File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/doctest.py", line 1231, in __run
        compileflags, 1) in test.globs
      File "<doctest couchdb.mapping[8]>", line 1, in <module>
        person = Person.load(db, person.id)
      File "/Users/christian/devel/carrot/couchdb-python-vanilla/couchdb/mapping.py", line 365, in load
        return cls.wrap(doc)
      File "/Users/christian/devel/carrot/couchdb-python-vanilla/couchdb/mapping.py", line 177, in wrap
        return cls(**data)
      File "/Users/christian/devel/carrot/couchdb-python-vanilla/couchdb/mapping.py", line 297, in __init__
        Mapping.__init__(self, **values)
      File "/Users/christian/devel/carrot/couchdb-python-vanilla/couchdb/mapping.py", line 137, in __init__
        setattr(self, attrname, values.pop(attrname))
      File "/Users/christian/devel/carrot/couchdb-python-vanilla/couchdb/mapping.py", line 104, in __set__
        value = self._to_json(value)
      File "/Users/christian/devel/carrot/couchdb-python-vanilla/couchdb/mapping.py", line 494, in _to_json
        value = datetime.combine(value, time(0))
    TypeError: combine() argument 1 must be datetime.date, not str
----------------------------------------------------------------------
File "/Users/christian/devel/carrot/couchdb-python-vanilla/couchdb/mapping.py", 
line 51, in couchdb.mapping
Failed example:
    person = Person.load(db, person.id)
Exception raised:
    Traceback (most recent call last):
      File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/doctest.py", line 1231, in __run
        compileflags, 1) in test.globs
      File "<doctest couchdb.mapping[15]>", line 1, in <module>
        person = Person.load(db, person.id)
      File "/Users/christian/devel/carrot/couchdb-python-vanilla/couchdb/mapping.py", line 365, in load
        return cls.wrap(doc)
      File "/Users/christian/devel/carrot/couchdb-python-vanilla/couchdb/mapping.py", line 177, in wrap
        return cls(**data)
      File "/Users/christian/devel/carrot/couchdb-python-vanilla/couchdb/mapping.py", line 297, in __init__
        Mapping.__init__(self, **values)
      File "/Users/christian/devel/carrot/couchdb-python-vanilla/couchdb/mapping.py", line 137, in __init__
        setattr(self, attrname, values.pop(attrname))
      File "/Users/christian/devel/carrot/couchdb-python-vanilla/couchdb/mapping.py", line 104, in __set__
        value = self._to_json(value)
      File "/Users/christian/devel/carrot/couchdb-python-vanilla/couchdb/mapping.py", line 494, in _to_json
        value = datetime.combine(value, time(0))
    TypeError: combine() argument 1 must be datetime.date, not str

======================================================================
FAIL: Doctest: couchdb.mapping.ListField
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/doctest.py", line 2131, in runTest
    raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for couchdb.mapping.ListField
  File "/Users/christian/devel/carrot/couchdb-python-vanilla/couchdb/mapping.py", line 578, in ListField

----------------------------------------------------------------------
File "/Users/christian/devel/carrot/couchdb-python-vanilla/couchdb/mapping.py", 
line 602, in couchdb.mapping.ListField
Failed example:
    post = Post.load(db, post.id)
Exception raised:
    Traceback (most recent call last):
      File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/doctest.py", line 1231, in __run
        compileflags, 1) in test.globs
      File "<doctest couchdb.mapping.ListField[8]>", line 1, in <module>
        post = Post.load(db, post.id)
      File "/Users/christian/devel/carrot/couchdb-python-vanilla/couchdb/mapping.py", line 365, in load
        return cls.wrap(doc)
      File "/Users/christian/devel/carrot/couchdb-python-vanilla/couchdb/mapping.py", line 177, in wrap
        return cls(**data)
      File "/Users/christian/devel/carrot/couchdb-python-vanilla/couchdb/mapping.py", line 297, in __init__
        Mapping.__init__(self, **values)
      File "/Users/christian/devel/carrot/couchdb-python-vanilla/couchdb/mapping.py", line 137, in __init__
        setattr(self, attrname, values.pop(attrname))
      File "/Users/christian/devel/carrot/couchdb-python-vanilla/couchdb/mapping.py", line 104, in __set__
        value = self._to_json(value)
      File "/Users/christian/devel/carrot/couchdb-python-vanilla/couchdb/mapping.py", line 494, in _to_json
        value = datetime.combine(value, time(0))
    TypeError: combine() argument 1 must be datetime.date, not str
----------------------------------------------------------------------
File "/Users/christian/devel/carrot/couchdb-python-vanilla/couchdb/mapping.py", 
line 603, in couchdb.mapping.ListField
Failed example:
    comment = post.comments[0]
Exception raised:
    Traceback (most recent call last):
      File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/doctest.py", line 1231, in __run
        compileflags, 1) in test.globs
      File "<doctest couchdb.mapping.ListField[9]>", line 1, in <module>
        comment = post.comments[0]
      File "/Users/christian/devel/carrot/couchdb-python-vanilla/couchdb/mapping.py", line 668, in __getitem__
        return self.field._to_python(self.list[index])
      File "/Users/christian/devel/carrot/couchdb-python-vanilla/couchdb/mapping.py", line 568, in _to_python
        return self.mapping.wrap(value)
      File "/Users/christian/devel/carrot/couchdb-python-vanilla/couchdb/mapping.py", line 177, in wrap
        return cls(**data)
      File "/Users/christian/devel/carrot/couchdb-python-vanilla/couchdb/mapping.py", line 137, in __init__
        setattr(self, attrname, values.pop(attrname))
      File "/Users/christian/devel/carrot/couchdb-python-vanilla/couchdb/mapping.py", line 104, in __set__
        value = self._to_json(value)
      File "/Users/christian/devel/carrot/couchdb-python-vanilla/couchdb/mapping.py", line 494, in _to_json
        value = datetime.combine(value, time(0))
    TypeError: combine() argument 1 must be datetime.date, not str
----------------------------------------------------------------------
File "/Users/christian/devel/carrot/couchdb-python-vanilla/couchdb/mapping.py", 
line 604, in couchdb.mapping.ListField
Failed example:
    comment['author']
Exception raised:
    Traceback (most recent call last):
      File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/doctest.py", line 1231, in __run
        compileflags, 1) in test.globs
      File "<doctest couchdb.mapping.ListField[10]>", line 1, in <module>
        comment['author']
    NameError: name 'comment' is not defined
----------------------------------------------------------------------
File "/Users/christian/devel/carrot/couchdb-python-vanilla/couchdb/mapping.py", 
line 606, in couchdb.mapping.ListField
Failed example:
    comment['content']
Exception raised:
    Traceback (most recent call last):
      File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/doctest.py", line 1231, in __run
        compileflags, 1) in test.globs
      File "<doctest couchdb.mapping.ListField[11]>", line 1, in <module>
        comment['content']
    NameError: name 'comment' is not defined
----------------------------------------------------------------------
File "/Users/christian/devel/carrot/couchdb-python-vanilla/couchdb/mapping.py", 
line 608, in couchdb.mapping.ListField
Failed example:
    comment['time'] #doctest: +ELLIPSIS
Exception raised:
    Traceback (most recent call last):
      File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/doctest.py", line 1231, in __run
        compileflags, 1) in test.globs
      File "<doctest couchdb.mapping.ListField[12]>", line 1, in <module>
        comment['time'] #doctest: +ELLIPSIS
    NameError: name 'comment' is not defined

----------------------------------------------------------------------
Ran 153 tests in 14.219s

FAILED (failures=2)

Original comment by christia...@orangelabs.at on 12 Jul 2011 at 5:23

Attachments:

GoogleCodeExporter commented 8 years ago 
As already mentioned same problem occurs when updating schema with a DictField. 
Attached patch (testcase and fix) for this bug. Fixes are also in my clone 
http://code.google.com/r/christianhaintz-couchdb-python/

Original comment by christia...@orangelabs.at on 12 Jul 2011 at 5:54

Attachments:

GoogleCodeExporter commented 8 years ago 
I've applied the last to patches on top of the current HEAD. Does not work for 
me.

import couchdb
from couchdb.mapping import *

class Alice(Document):
    field = ListField(TextField(), name='doesnotexist')

class Bob(Document):
    field = DictField(name='doesnotexist')

db = couchdb.Database('http://tom.iriscouch.com/test')
docid = "0578a01e-4ddc-45a7-8e15-a450dd97c213"

a = Alice.load(db, docid)
b = Bob.load(db, docid)

type(a.field)
# list

type(b.field)
# dict

a.field.append("foo")
# a = []

b.field['x'] = 'x'
# b = {}

Original comment by nschmuec...@gmail.com on 16 Dec 2011 at 12:08

GoogleCodeExporter commented 8 years ago 
Of course I mean "# a.field == []" and "# b.field == {}" in the last two 
comments.

Original comment by nschmuec...@gmail.com on 16 Dec 2011 at 12:09

GoogleCodeExporter commented 8 years ago 
This issue has been migrated to GitHub. Please continue discussion here:

https://github.com/djc/couchdb-python/issues/88

Original comment by djc.ochtman on 15 Jul 2014 at 7:17