Switch to NDB

[GoogleCodeExporter]

Follow up to issue #348 (which is about switching to Python 2.7).

What steps will reproduce the problem?
The problem was that Python 2.5 is outdated and we miss optional features from 
Python 2.7 (multi-threading) that potentially increase performance and coding 
speed.

It was believed (by me at least) that Python 2.7 is required to switch to NDB 
and get rid of caching clutches we've implementing in static classes. NDB comes 
with caching included. But it appeared that NDB works with 2.5 as well (and 
Python 2.7 doesn't require NDB or multi-threading to work).
https://groups.google.com/d/msg/codereview-list/R14YbZU1Qdo/Z9PR0foLjQsJ

So a switch to NDB will definitely help to turn on threading and clean up the 
code.

What is the expected output? What do you see instead?
As Guido mentioned "where parents and or references/keys are used to link 
between entities it's better to stick to one library". But some models (like 
Account?) doesn't seem to be linked. It would be nice to see graph of all 
Rietveld models though to make sure.

Is there a way to extract model relationships (graph) from the configured 
database layer at run-time in AppEngine and NDB?

Original issue reported on code.google.com by techtonik@gmail.com on 4 Jul 2012 at 3:15

• Blocked on: #429

[GoogleCodeExporter]

I've already prepared a quick'n'dirty diagram I've just shared with you both. 
I'll post the public link later when I'm back at my laptop.

Original comment by albrecht.andi on 4 Jul 2012 at 5:20

[GoogleCodeExporter]

I've noticed
https://docs.google.com/drawings/d/1xO7ZNu53U0cf04fX7v50xrpnzP-6OQomuDUT3VZMCX0/
edit opened in a new tab a several minutes after filling this issue.

I was actually looking for a way to export DB connection data to build diagrams 
automatically - see http://techtonik.bitbucket.org/modulegraph/index.html for 
an example of module connections.

This example is better - http://dhotson.github.com/springy/demo.html

Original comment by techtonik@gmail.com on 4 Jul 2012 at 5:36

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

Has anyone taken a stab at this?

Original comment by dhermes@google.com on 22 Jan 2013 at 2:44

[GoogleCodeExporter]

AFAIK, not yet.

Original comment by albrecht.andi on 24 Jan 2013 at 7:30

[GoogleCodeExporter]

A tool to inspect AppEngine models visually has more priority for me than a 
manual NDB migration.

Original comment by techtonik@gmail.com on 24 Jan 2013 at 10:52

[GoogleCodeExporter]

Some parent reference code snippet about parents for models:

from google.appengine.ext import db
from google.appengine.ext import ndb

class A(db.Model):
  a = db.StringProperty()

class B(db.Model):
  b = db.StringProperty()

a = A(a='10')
a.put()
A_ID = a.key().id()

b = B(b='11', parent=a)
b.put()
B_ID = b.key().id()

class A(ndb.Model):
  a = ndb.StringProperty()

class B(ndb.Model):
  b = ndb.StringProperty()

a = A.get_by_id(A_ID)
b = B.get_by_id(B_ID, parent=a.key)
print b

Original comment by dhermes@google.com on 11 Mar 2013 at 7:04

[GoogleCodeExporter]

Some (should be comprehensive for our models.py) snippets for reference 
properties:

from google.appengine.ext import db
from google.appengine.ext import ndb

class A(db.Model):
  val = db.StringProperty()

class B(db.Model):
  a = db.ReferenceProperty(A)
  b = db.SelfReferenceProperty()
  a_collection = db.ReferenceProperty(A, collection_name='bees')

a1 = A(val='10')
a1.put()

b1 = B(a=a1, a_collection=a1)
b1.put()

a2 = A(val='11')
a2.put()

b2 = B(a=a2, b=b1, a_collection=a1)
b2.put()

for b in a1.bees:
  print b.key().id()

A1_ID = a1.key().id()
A2_ID = a2.key().id()

B1_ID = b1.key().id()
B2_ID = b2.key().id()

class A(ndb.Model):
  val = ndb.StringProperty()

  @property
  def bees(self):
    return B.query(B.a_collection == self.key)

class B(ndb.Model):
  a = ndb.KeyProperty(kind=A)
  b = ndb.KeyProperty(kind='B')
  a_collection = ndb.KeyProperty(kind=A)

a1 = A.get_by_id(A1_ID)
a2 = A.get_by_id(A2_ID)

print a1
print a2

b1 = B.get_by_id(B1_ID)
b2 = B.get_by_id(B2_ID)

print b1
print b2

for b in a1.bees:
  print b

Original comment by dhermes@google.com on 14 Mar 2013 at 3:42

[GoogleCodeExporter]

Original comment by albrecht.andi on 16 Mar 2013 at 8:43

• Now blocked on: #429

[GoogleCodeExporter]

I had written a CL switching Repository and Branch but my HD failed before I 
completed it (I wish I was kidding). I already asked dhermes to do preliminary 
work and he added NdbAnyAuthUserProperty in 
https://code.google.com/p/rietveld/source/detail?r=ba438e8e3378.

I'd recommend doing these 2 entities first since they are simple and not too 
much used, and then go with the rest of the models.

Original comment by maruel@chromium.org on 16 Oct 2013 at 4:53

[GoogleCodeExporter]

This has been done.

Original comment by jrobbins@google.com on 11 Jul 2014 at 11:37

• Changed state: Fixed