Parent/Child relationship requires manually assignment of parent ID/Key

[GoogleCodeExporter]

I am trying to define a simple parent/child relationship. The child class will 
not define its Parent because the Parent could be one of many kinds. When I 
annotate the ChildModel in the ParentModel with @Child, I get an exception 
complaining about the Key being incomplete. I found that by specifying a the 
value of the field @Id with some value, this exception goes away and my test 
passes.

The reason I am opening this as a bug, is because I do not (should not) want to 
specify the key manually. This behavior is also inconsistent with GAE JDO 
behavior. When I was using JDO, I was able to have the @Id managed by the 
datastore.

This is using the Twig 2.0-beta3 jar and GAE 1.3.8

public class ParentModel {
        // I dont want to have to specify this
        //@Id
        //protected Long dummyId;

    protected String dummyProperty1;

    @Child
    protected ChildModel child = new ChildModel();
}

public class ChildModel {

    protected String dummyProperty1;
}

// Failing test:
ParentModel p = new ParentModel();
datastore.store().instance(p).now();

Stacktrace:
java.lang.IllegalStateException: Key specification is incomplete.  You may need 
to define an id for instance with kind com_schedgy_data_ParentModel
        at com.google.code.twig.standard.KeySpecification.toKey(KeySpecification.java:90)
        at com.google.code.twig.standard.ChildRelationTranslator.getParentKey(ChildRelationTranslator.java:19)
        at com.google.code.twig.standard.RelationTranslator.instanceToKey(RelationTranslator.java:164)
        at com.google.code.twig.standard.RelationTranslator$2.get(RelationTranslator.java:151)
        at com.google.code.twig.standard.RelationTranslator$2.get(RelationTranslator.java:148)
        at com.google.code.twig.standard.StandardEncodeCommand.dereferencePropertyValue(StandardEncodeCommand.java:58)
        at com.google.code.twig.standard.StandardEncodeCommand.transferProperties(StandardEncodeCommand.java:41)
        at com.google.code.twig.standard.StandardCommonStoreCommand.instanceToEntity(StandardCommonStoreCommand.java:252)
        at com.google.code.twig.standard.StandardSingleStoreCommand.now(StandardSingleStoreCommand.java:51)
        at com.google.code.twig.standard.StandardSingleStoreCommand.now(StandardSingleStoreCommand.java:13)
        at com.schedgy.data.TwigApiTest.testStoreParentChild(TwigApiTest.java:43)

Original issue reported on code.google.com by barbedwi...@gmail.com on 6 Nov 2010 at 9:16

[GoogleCodeExporter]

This seems like a pretty straightforward and necessary capability. I ran into 
the same problem. Would very much appreciate some feedback on this issue. Is it 
not possible to have the system generate an ID for a top level object? I have a 
Person object which references a UserPwd object (contains login information), 
an Address object (contains mailing address info), and a collection of IRole 
(interface) objects (contains roles played by this Person). I get the same 
behavior described above and need a solution to this. I don't see how I can 
justify the use of twig if it is not possible to have the system generate an ID 
for the top level object. Please help. Thanks.

Original comment by rickh...@gmail.com on 25 Mar 2011 at 2:28

[GoogleCodeExporter]

This is a chicken and egg problem that I am pretty sure JDO does not handle.

You see, the Person contains the Key for the UserPwd so it must have that key 
before it can be stored.  If the UserPwd is a child of the Person then its key 
actually contains the key of the Person which.... hasn't been stored yet.  So 
the Person must have its @Id set before being stored - then the UsePwd can know 
the Key of the parent (Person) without storing it.

Luckily, help is at hand by using the annotation:

@Entity(allocateIdsBy=10)
public class Person {...}

That will tell Twig to get ids from the datastore (unique) and set the 
Person.id.

It is not very well tested yet, but hopefully this has explained why the 
fundamental problem exists.  I am not sure that any other frameworks provide an 
equivalent.

Original comment by jdpatterson on 25 Mar 2011 at 3:41

• Changed state: Fixed

[GoogleCodeExporter]

Thank you for the quick response. 

Unless I'm mistaken, JDO does support this. I'm trying to convert an existing 
JDO/GAE application. In my existing application, I have several places in which 
I store a top-level object that contains child objects within the same entity 
group. 

More importantly, I still cannot get this to work. I'm running twig-persist 
2.0b4, and get the following exception when trying to store Person with the key 
annotated as:
@Id String key;
where key == null;

Caused by: java.lang.NullPointerException
    at com.google.code.twig.standard.StandardCommonStoreCommand.maybeSetAllocatedId(StandardCommonStoreCommand.java:290)
    at com.google.code.twig.standard.StandardCommonStoreCommand.instanceToEntity(StandardCommonStoreCommand.java:251)
    at com.google.code.twig.standard.StandardSingleStoreCommand.now(StandardSingleStoreCommand.java:51)
    at com.google.code.twig.standard.TranslatorObjectDatastore.store(TranslatorObjectDatastore.java:144)
    at com.toolcafe.gwtstore.server.AbstractAdminServiceImpl.createBaseAdmin(AbstractAdminServiceImpl.java:65)
    at com.youvoiceit.server.AdminServiceImpl.initServer(AdminServiceImpl.java:20)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:616)
    at com.google.appengine.tools.development.agent.runtime.Runtime.invoke(Runtime.java:100)
    at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:569)
    ... 30 more

I thought that perhaps I needed the latest sources, so I "cloned" the source to 
my system and tried to build. I get some compile errors and not sure how to fix 
them. To build, I copied the sources to an Eclipse Java project, added the 
Google Collections JAR, junit-4.9b2, appengine-testing.jar, and referenced the 
Google App Engine library. The errors are as follows:

Description Resource    Path    Location    Type
Description Resource    Path    Location    Type
The constructor QueryResultIteratorImpl(AsyncPreparedQuery, List<Entity>, 
QueryResultsSourceImpl, FetchOptions, Transaction) is 
undefined   AsyncPreparedQuery.java /twig-persist/src/com/google/appengine/api/dat
astore  line 97 Java Problem
The constructor QueryResultsSourceImpl(null, FetchOptions, Transaction) is 
undefined   AsyncPreparedQuery.java /twig-persist/src/com/google/appengine/api/dat
astore  line 95 Java Problem
The method loadFromPb(DatastorePb.QueryResult) is undefined for the type 
QueryResultsSourceImpl  AsyncPreparedQuery.java /twig-persist/src/com/google/appe
ngine/api/datastore line 96 Java Problem
The method run() of type new TransactionRunner(){} must override or implement a 
supertype 
method  AsyncDatastoreHelper.java   /twig-persist/src/com/google/appengine/api/data
store   line 38 Java Problem
The type AsyncPreparedQuery must implement the inherited abstract method 
PreparedQuery.countEntities(FetchOptions)   AsyncPreparedQuery.java /twig-persist/
src/com/google/appengine/api/datastore  line 15 Java Problem
The type new TransactionRunner(){} must implement the inherited abstract method 
TransactionRunner.runInternal() AsyncDatastoreHelper.java   /twig-persist/src/com/
google/appengine/api/datastore  line 35 Java Problem

Thanks,

Rick

Original comment by rickh...@gmail.com on 25 Mar 2011 at 7:10

[GoogleCodeExporter]

I haven't yet gotten a clean build of twig-persist, as described above, but I 
decided to do more testing with the latest sources. I tried two alternatives: 

Person with @Id annotation associated with the key
@Id String key;
and without the annotation:
String key;

In both cases, I get the same Exception stated in my previous post.

Caused by: java.lang.NullPointerException
    at com.google.code.twig.standard.StandardCommonStoreCommand.maybeSetAllocatedId(StandardCommonStoreCommand.java:290)
    at com.google.code.twig.standard.StandardCommonStoreCommand.instanceToEntity(StandardCommonStoreCommand.java:251)
    at com.google.code.twig.standard.StandardSingleStoreCommand.now(StandardSingleStoreCommand.java:51)
    at com.google.code.twig.standard.TranslatorObjectDatastore.store(TranslatorObjectDatastore.java:144)

Original comment by rickh...@gmail.com on 25 Mar 2011 at 7:46

[GoogleCodeExporter]

The latest source has a fix for this issue.  Can you check it out to include in 
your project?

Original comment by jdpatterson on 26 Mar 2011 at 3:10

[GoogleCodeExporter]

Sorry, JD. Still not working. I continue to get the same exception when I try 
to store the Person object referencing a UserPwd, Administrator (via a 
collection), and an Address -- when Person's @Id private String key == null. 
I'm probably doing something wrong, but not sure what.

Caused by: java.lang.NullPointerException
    at com.google.code.twig.standard.StandardCommonStoreCommand.maybeSetAllocatedId(StandardCommonStoreCommand.java:290)
    at com.google.code.twig.standard.StandardCommonStoreCommand.instanceToEntity(StandardCommonStoreCommand.java:251)
    at com.google.code.twig.standard.StandardSingleStoreCommand.now(StandardSingleStoreCommand.java:51)
    at com.google.code.twig.standard.TranslatorObjectDatastore.store(TranslatorObjectDatastore.java:144)

Would you like me to send you the relevant code? If so, what's the best way to 
do so?

Original comment by rickh...@gmail.com on 26 Mar 2011 at 3:21

[GoogleCodeExporter]

First, are you sure you have the latest source from hg?  Do you perhaps have a 
jar in the /WEB-INF/lib dir?

Line 290 is where the fix went in and I can't see that it could possibly throw 
a NPE:

http://code.google.com/p/twig-persist/source/browse/src/main/java/com/google/cod
e/twig/standard/StandardCommonStoreCommand.java?r=v2.0#290

Original comment by jdpatterson on 26 Mar 2011 at 4:26

[GoogleCodeExporter]

Its always the classpath! :-) Yes, I forgot to remove the old JAR! My 
apologies. BUT, it still isn't working. When I set the Person's id field prior 
to calling store(), store() works, but when I do not, I get the following 
exception:

Caused by: java.lang.IllegalStateException: Key specification is incomplete.  
You may need to define a key name for this or its parent instance
    at com.vercer.engine.persist.standard.KeySpecification.toKey(KeySpecification.java:96)
    at com.vercer.engine.persist.standard.ChildEntityTranslator.getParentKey(ChildEntityTranslator.java:19)
    at com.vercer.engine.persist.standard.EntityTranslator$2.get(EntityTranslator.java:121)
    at com.vercer.engine.persist.standard.EntityTranslator$2.get(EntityTranslator.java:1)
    at com.vercer.engine.persist.standard.StrategyObjectDatastore.dereferencePropertyValue(StrategyObjectDatastore.java:987)
    at com.vercer.engine.persist.standard.StrategyObjectDatastore.transferProperties(StrategyObjectDatastore.java:970)
    at com.vercer.engine.persist.standard.StrategyObjectDatastore.instanceToEntity(StrategyObjectDatastore.java:920)
    at com.vercer.engine.persist.standard.StrategyObjectDatastore.internalStore(StrategyObjectDatastore.java:668)
    at com.vercer.engine.persist.standard.StrategyObjectDatastore.store(StrategyObjectDatastore.java:658)
    at com.vercer.engine.persist.standard.StrategyObjectDatastore.store(StrategyObjectDatastore.java:1011)
    at com.toolcafe.gwtstore.server.AbstractAdminServiceImpl.createBaseAdmin(AbstractAdminServiceImpl.java:69)

Please see the attached file for a segment of Person and my code used to store 
it.

Original comment by rickh...@gmail.com on 26 Mar 2011 at 5:32

Attachments:

• TwigErrFile.txt

[GoogleCodeExporter]

Well, progress is being made at least.  Its a new exception :)

I had only briefly tested this with Child instances - not parent instances.

Seeing as I don't have a test case yet for this (hint hint patch welcome) could 
you apply this patch and see if it fixes your problem.  It basically just moves 
a single line

diff -r 6d844455e808 
src/main/java/com/google/code/twig/standard/StandardCommonStoreCommand.java
--- 
a/src/main/java/com/google/code/twig/standard/StandardCommonStoreCommand.java   Th
u Mar 24 14:25:25 2011 +0700
+++ 
b/src/main/java/com/google/code/twig/standard/StandardCommonStoreCommand.java   Su
n Mar 27 00:49:24 2011 +0700
@@ -156,7 +156,7 @@
        Map<T, Entity> entities = new LinkedHashMap<T, Entity>(instances.size());
        if (batch)
        {
-           // indicates that all entities should be collected for one put
+           // all entities will  be collected for one bulk put
            datastore.batched = (Map<Object, Entity>) entities;
        }

@@ -239,6 +239,8 @@
            datastore.keyCache.cacheKeyReferenceForInstance(instance, datastore.encodeKeySpec.toObjectReference());
        }

+       maybeSetAllocatedId(instance);
+
        // translate fields to properties - sets key parent and id
        PropertyTranslator encoder = datastore.encoder(instance);
        Set<Property> properties = encoder.encode(instance, Path.EMPTY_PATH, datastore.indexed);
@@ -246,8 +248,6 @@
        {
            throw new IllegalStateException("Could not translate instance: " + instance);
        }
-
-       maybeSetAllocatedId(instance);

        // the key will now be set with id and parent
        Entity entity = createEntity();

Original comment by jdpatterson on 26 Mar 2011 at 5:51

[GoogleCodeExporter]

btw, I'm liking twig very much. It looks like I'll be able to remove lots of 
JDO cruft (due to the feeble GAE implementation). Thank you for helping with 
this problem. 

I'm confused, though. Your patch references:

com/google/code/twig/standard/StandardCommonStoreCommand.java

whereas the source code I've downloaded uses package:

com.vercer.engine.persist.*

The code I downloaded came from:

http://code.google.com/p/twig-persist/source/browse/#hg%2Fsrc%2Fshared%2Fcom%2Fv
ercer

via: 
hg clone https://twig-persist.googlecode.com/hg/ twig-persist

Do I have the right code?

Thanks,

Rick

Original comment by rickh...@gmail.com on 26 Mar 2011 at 8:58

[GoogleCodeExporter]

Ah-ha!! You've got the v1.0 branch :)

When you do your "pull" you have to choose "v2.0" from the small drop down - 
not very clear unfortunately.  Give that a shot and get back to me.

Original comment by jdpatterson on 27 Mar 2011 at 4:15

[GoogleCodeExporter]

Sorry that drop down is on the google site for browsing the code.  When you 
pull the code you get all changesets (both branches) and you need to change 
branch.

hg branches
hg branch v2.0

I've just checked in a test case that shows allocateIdsBy working on a parent 
class:

http://code.google.com/p/twig-persist/source/browse/src/test/java/com/google/cod
e/twig/tests/unit/StoreCommandTest.java

Original comment by jdpatterson on 27 Mar 2011 at 6:09

[GoogleCodeExporter]

John, we're making progress. I figured the latest code would be the default, 
but assuming sometimes gets you in trouble. :-) I now have the V2.0 code, built 
with Maven, and have done some more testing. twig is very cool, very nice 
programming model. I have uncovered a couple of problems. Appreciate it if you 
could look into these.

1. I still can't get automatic id generation working. The reason appears to be 
that I am using a String as the ID type. Here's the exception.

Caused by: java.lang.IllegalStateException: You must set an id value if the id 
field is not a numeric type.
    at com.google.code.twig.standard.StandardCommonStoreCommand.setInstanceId(StandardCommonStoreCommand.java:103)
    at com.google.code.twig.standard.StandardSingleStoreCommand.now(StandardSingleStoreCommand.java:60)
    at com.google.code.twig.standard.TranslatorObjectDatastore.store(TranslatorObjectDatastore.java:145)
    at com.toolcafe.gwtstore.server.AbstractAdminServiceImpl.createBaseAdmin(AbstractAdminServiceImpl.java:69)

I'd like to keep by IDs defined as Strings (unless there's a really good reason 
not to) as I have quite a lot of code dependent on these definitions, and don't 
really want to hack them all out. Making String IDs work seems like it would be 
a straightforward fix. Is that the case, and if so, can you please fix this?

2. @Entity doesn't seem to honor the "kind" attribute. I have the following 
code:

@Entity(allocateIdsBy=10, kind="Person")

...but the entity kind generated contains the Java package name. Seems like a 
bug.

Thanks again,

Rick

Original comment by rickh...@gmail.com on 27 Mar 2011 at 4:52

[GoogleCodeExporter]

in reverse order...

2. yes this is an incomplete feature sorry.  I'm working on getting this done 
in a branch that has a few broken bits.  If you create an issue for it you will 
be notified as soon as it is checked in.  For now please use 
DefaultConfiguration.registerTypeName(Class, String)

I am working on a DefaultConfiguration.register(Classs) and have plans to add a 
package scanner that will discover entity classes.

1. Not really... you see the id is set normally after being generated on the 
server (always a long).  Previously the code did what you are asking: it would 
convert to String.  But then when using the instance again it would not be 
found because the auto-generated id e.g. 5 != "5".  This caused a subtle bug 
where update() created a duplicate entity.

To add this ability would require special case code for pre-allocated ids.  
Probably not worth the complication when you can easily do the conversion to 
String in your data model getId():String.  Remember Twig access fields 
directly, not though setters.

Original comment by jdpatterson on 27 Mar 2011 at 5:46