search

GoogleCloudPlatform / endpoints-proto-datastore

Apache License 2.0
154 stars 52 forks source link

Add support for NDB polymodel #28

Open dhermes opened 11 years ago

dhermes commented 11 years ago

Try to add support for polymodel https://developers.google.com/appengine/docs/python/ndb/polymodelclass

Need to investigate/discuss before fully committing.

Code Hosting Comment Metadata: sub>author.htmlLink=https://code.google.com/u/dhermes@google.com/</sub sub>author.name=dhermes@google.com</sub id=0 published=2013-01-10T01:53:13.000Z

Code Hosting Issue Metadata: published=2013-01-10T01:53:13.000Z stars=2 updated=2013-04-23T18:21:34.000Z originalIssue=https://code.google.com/p/endpoints-proto-datastore/issues/detail?id=32 sub>author=@dhermes</sub

dhermes commented 11 years ago

A few other missing things which may be worth pursuing:

stats.py:

class BaseStatistic(model.Model):

polymodel.py:

class _ClassKeyProperty(model.StringProperty):
class PolyModel(model.Model):

msgprop.py:

class EnumProperty(model.IntegerProperty):
class MessageProperty(model.StructuredProperty):

metadata.py:

class _BaseMetadata(model.Model):

blobstore.py:

class BlobInfo(model.Model):

NOTE: THIS POST WAS EDITED AFTER BEING MIGRATED.

Code Hosting Comment Metadata: sub>author.htmlLink=https://code.google.com/u/dhermes@google.com/</sub sub>author.name=dhermes@google.com</sub id=1 published=2013-01-10T04:57:04.000Z

dhermes commented 11 years ago

This can be made to work with a few tweaks, mainly in an _EndpointsQueryInfo container class and in EndpointsModel.ToMessageCollection.

A full sample which extends the basic sample:

from google.appengine.ext import ndb
from google.appengine.ext.ndb import polymodel
from protorpc import remote

import endpoints

from endpoints_proto_datastore.ndb import EndpointsModel
from endpoints_proto_datastore.ndb.model import _EndpointsQueryInfo

def _DowncastMessage(message, final_message_class):
  message_class = message.__class__

  downcasted_message = final_message_class()
  for field in final_message_class.all_fields():
    # KeyError allowed to happen, if the field is missing,
    # a downcast should be performed
    field_on_message = message_class.field_by_name(field.name)
    # Make sure fields are the same type
    if field_on_message.__class__ != field.__class__:
      raise TypeError('Downcasted field of the wrong type: %r. Should be %r.' %
                      (field_on_message.__class__, field.__class__))
    value = getattr(message, field.name)
    setattr(downcasted_message, field.name, value)
  return downcasted_message

class _PolyModelQueryInfo(_EndpointsQueryInfo):

  def _PopulateFilters(self):
    entity = self._entity
    for prop in entity._properties.itervalues():
      if isinstance(prop, polymodel._ClassKeyProperty):
        continue
      attr_name = prop._code_name
      current_value = getattr(entity, attr_name)

      # Only filter for non-null values
      if current_value is not None:
        self._AddFilter(prop == current_value)

class MyModel(EndpointsModel, polymodel.PolyModel):
  _message_fields_schema = ('attr1', 'created')

  attr1 = ndb.StringProperty()
  created = ndb.DateTimeProperty(auto_now_add=True)

  def __init__(self, *args, **kwargs):
    # Don't need to call both constructors since PolyModel doesn't define one
    # and descends from model.Model, a superclass of EndpointsModel
    super(MyModel, self).__init__(*args, **kwargs)
    self._endpoints_query_info = _PolyModelQueryInfo(self)

  @classmethod
  def ToMessageCollection(cls, items, collection_fields=None,
                          next_cursor=None):
    proto_model = cls.ProtoCollection(collection_fields=collection_fields)

    items_as_message = [item.ToMessage(fields=collection_fields)
                        for item in items]
    final_proto_class = cls.ProtoModel(fields=collection_fields)
    items_as_message = [_DowncastMessage(item, final_proto_class)
                        for item in items_as_message]

    result = proto_model(items=items_as_message)

    if next_cursor is not None:
      result.nextPageToken = next_cursor.to_websafe_string()

    return result

class NextModel(MyModel):
  _message_fields_schema = ('attr1', 'attr2', 'created')
  attr2 = ndb.StringProperty()

@endpoints.api(name='myapi', version='v1', description='My Little API')
class MyApi(remote.Service):

  @MyModel.method(path='mymodel', http_method='POST', name='mymodel.insert')
  def MyModelInsert(self, my_model):
    my_model.put()
    return my_model

  @MyModel.query_method(path='mymodels', name='mymodel.list')
  def MyModelList(self, query):
    return query

  @NextModel.method(path='nextmodel', name='nextmodel.insert')
  def NextModelInsert(self, my_model):
    my_model.put()
    return my_model

  @NextModel.query_method(path='nextmodels', name='nextmodel.list')
  def NextModelList(self, query):
    return query

application = endpoints.api_server([MyApi], restricted=False)

NOTE: THIS POST WAS EDITED AFTER BEING MIGRATED.

Code Hosting Comment Metadata: sub>author.htmlLink=https://code.google.com/u/dhermes@google.com/</sub sub>author.name=dhermes@google.com</sub id=2 published=2013-01-10T09:11:00.000Z

dhermes commented 11 years ago

After discussing, we have decided to write a subclass EndpointsPolyModel which performs most of the things above and extends EndpointsModel and polymodel.PolyModel in the same fashion.

As of right now, it's unclear when. Feel free to star the issue if it is urgent for you or if you are using the code from the comment above.

NOTE: THIS POST WAS EDITED AFTER BEING MIGRATED.

Code Hosting Comment Metadata: sub>author.htmlLink=https://code.google.com/u/dhermes@google.com/</sub sub>author.name=dhermes@google.com</sub id=3 published=2013-01-10T18:05:26.000Z updates.labels=[   -Type-Discussion,   Type-FeatureRequest, ] updates.status=Accepted updates.summary=Add support for polymodel

dhermes commented 11 years ago

If users wanted a generic query for an ID to return the model with all properties, this may be worth offering. For example:

class Event(EndpointsModle, polymodel.PolyModel):
  # all the other stuff needed from http://pastebin.com/hdj4eqTY
  attr1 = ndb.StringProperty()

class UserAddedEvent(Event):
  attr2 = ndb.IntegerProperty()

class RegistrationEvent(Event):
  attr3 = ndb.FloatProperty()

To be able to return an instance of any of these, we'd need

from protorpc import messages
class EventUmbrella(messages.Message):
  attr1 = messages.StringField(1)
  attr2 = messages.IntegerField(2)
  attr3 = messages.FloatField(3)
  # Maybe a string field to indicate the type returned
  event_class = messages.StringField(4)  # can't use class as a varname in Python

And then use the protorpc class instead of response_fields:

  @Event.method(..., response_message=EventUmbrella, ...)
  def simple_get(self, entity):
    # ... do stuff to entity
    umbrella_message = ConvertToEventUmbrella(entity)
    return umbrella_message

ToMessageCollection would also need to be tweaked on the base EndpointsPolyModel class to allow this umbrella class instead of whatever is implicitly used by ProtoCollection.

Also, it may be worth offering a method which such that

f(Event, UserAddedEvent, RegistrationEvent) = EventUmbrella

Link to Full Discussion: https://groups.google.com/forum/?fromgroups=#!topic/endpoints-trusted-testers/gd3zXk6XkZM

NOTE: THIS POST WAS EDITED AFTER BEING MIGRATED.

Code Hosting Comment Metadata: sub>author.htmlLink=https://code.google.com/u/dhermes@google.com/</sub sub>author.name=dhermes@google.com</sub id=4 published=2013-01-11T05:57:13.000Z

dhermes commented 11 years ago

Code Hosting Comment Metadata: sub>author.htmlLink=https://code.google.com/u/dhermes@google.com/</sub sub>author.name=dhermes@google.com</sub id=5 published=2013-02-23T06:29:43.000Z updates.summary=Add support for NDB polymodel

dhermes commented 11 years ago

NDB polymodel exists now: https://developers.google.com/appengine/docs/python/ndb/polymodelclass

Code Hosting Comment Metadata: author.htmlLink=https://code.google.com/u/108103421167573850503/ sub>author.name=james@khanacademy.org</sub id=6 published=2013-04-23T17:48:12.000Z

dhermes commented 11 years ago

Thanks James.

Unfortunately, that's not what this bug is about :)

It's about whether or not it's worth it/possible to support this in endpoints-proto-datastore.

Code Hosting Comment Metadata: sub>author.htmlLink=https://code.google.com/u/dhermes@google.com/</sub sub>author.name=dhermes@google.com</sub id=7 published=2013-04-23T18:21:34.000Z

decurgia commented 10 years ago

Hi

How can I star this issue? I would love to see this feature as I plan to use a Polymodel datastore model. The model shall look like this:

dhermes commented 10 years ago

Have you tried some of the suggestions above?

tmst commented 6 years ago

I take it that MyModel::list not returning the attr2 property of NextModel entities is addressed, above, and that the implementation is not trivial. Would I be correct in this?