List items from repeated StringProperty get mutated to _BaseValue on put

[GoogleCodeExporter]

Items in a list retrieved from a repeated StringProperty get mutated from 
string to _BaseValue when 'put' is called on the entity. If the property is 
read, the items are mutated back to strings.

This behaviour was observed in the Python SDK (version 1.7.1).

import logging
from google.appengine.ext.ndb import model

class MyModel(model.Model):
    items = model.StringProperty(repeated=True)

prepare_entity = MyModel(id='test', items=['abc'])
prepare_entity.put()

entity = MyModel.get_by_id(id='test')
domain_items = entity.items
assert isinstance(domain_items[0], basestring)

entity.put()
entity.items
assert isinstance(domain_items[0], basestring)

entity.put()
assert isinstance(domain_items[0], basestring)

Original issue reported on code.google.com by pe...@mtrotter.com on 19 Sep 2012 at 6:54

[GoogleCodeExporter]

You are mostly describing how the implementation works for all values. Which if 
the asserts in your example program is failing?

Original comment by guido@google.com on 19 Sep 2012 at 7:05

[GoogleCodeExporter]

The final assert fails, since the value in the list at that point is a 
_BaseValue.

Original comment by pe...@mtrotter.com on 19 Sep 2012 at 7:09

[GoogleCodeExporter]

I see. We should probably document that you're not supposed to hang on to the 
list of items for too long; there are various forms of undefined behavior 
around that. IIRC you also shouldn't do things like

items = ['x']
ent = MyModel(items=items)
items.append('y')
# At this point, is ent.items the same as items?

Original comment by guido@google.com on 19 Sep 2012 at 7:13

[GoogleCodeExporter]

My naive mental model was that ent.items and domain_items referred to the same 
list. In your example

items = ['x']
ent = MyModel(items=items)
items.append('y')

items would be passed by reference, and modifications to both ent.items and 
items would be shared. I was suprised that 'items' was mutated when it was 
'put' since I expected that to remain an internal affair of NDB.

Documentation would certainly help.

Original comment by pe...@mtrotter.com on 19 Sep 2012 at 7:36

[GoogleCodeExporter]

I was just bitten by this as well and it was hard to track because I didn't 
expect my lists to change on me. I feel that NDB should do it's internal work 
on a separate copy of the list to avoid messing it up for unsuspecting callers.

Original comment by wal...@ninua.com on 3 May 2013 at 1:28

[GoogleCodeExporter]

I was just hit by this is as well, and I have to agree with wal.

Preferably NDB should do its transformation on an internal copy (or similar to 
achieve the same effect), but minimally this should be mentioned in the 
documentation.

My workaround involves always copying lists from NDB:
https://github.com/web2py/web2py/pull/316

Original comment by anssi.hannula@gmail.com on 2 Dec 2013 at 9:36

[GoogleCodeExporter]

I am working on a ndb system and was bitten by this too. hard to track and very 
annoying.

Original comment by tiago.fr...@cert-id.com.br on 23 Jan 2014 at 11:10

[GoogleCodeExporter]

doing a second look at my code, I am reading values from the database and 
copying my data using to_dict() BEFORE the put() command. 
After the put() I am trying to use json.dumps(), but I am getting an error on 
_BaseValue.

Original comment by tiago.fr...@cert-id.com.br on 23 Jan 2014 at 11:24

[GoogleCodeExporter]

It's like this pseudocode:

(get or create ndb_entity_instance)
(set values)
varcopy = ndb_entity_instance.to_dict()
logging.info(varcopy) #good, no _BaseValues
ndb_entity_instance.put()
logging.info(varcopy) #BAD, _BaseValues all around

Original comment by tiago.fr...@cert-id.com.br on 23 Jan 2014 at 11:29

[GoogleCodeExporter]

Original comment by pcoste...@google.com on 24 Feb 2014 at 9:20

• Changed state: Accepted
• Added labels: log-13171738