Throw exception when filtering or sorting by pk-id or pk-name fields

[GoogleCodeExporter]

WHAT STEPS WILL REPRODUCE THE PROBLEM?
--------------------------------------
Under Java and JDO and using the persistent entities and code skeleton as
given below, persist a new child Item to belong to an existing parent User.

WHAT IS THE EXPECTED OUTPUT? WHAT DO YOU SEE INSTEAD?
-----------------------------------------------------
  * The new item can be found using a query on all items where the correct
user is the parent (as expected).
  * The user's list of items is empty (I expect this to be populated with
one Item). Sometimes on first access it can be non-empty but not
subsequently, as in

  int n = -1;
  if (!user.getItems().isEmpty())
    n = user.getItems().size();

where n can be end up as zero! The list of items is non-empty, but on the
next line becomes, and then remains, empty.

WHAT VERSION OF THE PRODUCT ARE YOU USING? ON WHAT OPERATING SYSTEM?
--------------------------------------------------------------------
GAE/J version: 1.3.0.

Operating system: Microsoft Windows XP Media Center Edition Version 2002
Service Pack 3

NetBeans development, build and run environment:
  Product Version: NetBeans IDE 6.7.1 (Build 200907230233)
  Java: 1.6.0_17; Java HotSpot(TM) Client VM 14.3-b01
  System: Windows XP version 5.1 running on x86; Cp1252; en_GB (nb)
  Userdir: [...]

Web browser: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.1.6)
Gecko/20091201 Firefox/3.5.6

ADDITIONAL INFORMATION
----------------------
If the definition of the parent side of the 1-N relationship is changed so
that the field used in the sort value is changed away from the ID part of
the encoded string primary key, then all works fine. For example, changing
'value="loID ASC"' to 'value="sEncodedKey ASC"' as shown in the following
amended field definition results in correct behaviour of the list:

  @Persistent(mappedBy="userParent")
  @Element(dependent="true")
  @Order(extensions = @Extension(vendorName="datanucleus", key="list-ordering",
   value="sEncodedKey ASC"))
  private ArrayList<Item> liItems = new ArrayList<Item>();

I do not know if this issue is related to datanucleus-appengine Issue 188
(Child object with custom pk column can't be saved).

JAVA CODE
---------
Listed below is the relevant portions of the code for the entity group root
persistent class "User" and its child entity class "Item". Following that
is the skeleton of the persistence code.

////////////////////////////////////////////////////////////////////////////////
// Parent entity class
//

@PersistenceCapable(identityType = IdentityType.APPLICATION,
 detachable = "true")
public class User implements Serializable
{
  private static final long serialVersionUID = 1L;

  @PrimaryKey
  @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
  @Extension(vendorName="datanucleus", key="gae.encoded-pk", value="true")
  private String sEncodedKey;

  @Persistent
  @Extension(vendorName="datanucleus", key="gae.pk-id", value="true")
  private Long loID;

  @Persistent(mappedBy="userParent")
  @Element(dependent="true")
  @Order(extensions = @Extension(vendorName="datanucleus", key="list-ordering",
   value="loID ASC"))
  private ArrayList<Item> liItems = new ArrayList<Item>();

  public String getEncodedKey()
  {
    return sEncodedKey;
  }

  public Long getID()
  {
    return loID;
  }

  public ArrayList<Item> getItems()
  {
    return liItems;
  }

  public void setItems(ArrayList<Item> items)
  {
    liItems = items;
  }

  @SuppressWarnings("unchecked")
  public void addItem(Item item)
  {
    if (liItems == null)
      liItems = new ArrayList<Item>();

    liItems.add(item);
  }
}

//
////////////////////////////////////////////////////////////////////////////////

////////////////////////////////////////////////////////////////////////////////
// Child entity class
//

@PersistenceCapable(identityType = IdentityType.APPLICATION,
 detachable = "true")
public class Item implements Serializable
{
  private static final long serialVersionUID = 1L;

  @PrimaryKey
  @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
  @Extension(vendorName="datanucleus", key="gae.encoded-pk", value="true")
  private String sEncodedKey;

  @Persistent
  @Extension(vendorName="datanucleus", key="gae.pk-id", value="true")
  private Long loID;

  @Persistent(dependent="false")
  private User userParent;

  public String getEncodedKey()
  {
    return sEncodedKey;
  }

  public Long getID()
  {
    return loID;
  }

  public User getUserParent()
  {
    return userParent;
  }

  public void setUserParent(User parent)
  {
    userParent = parent;
  }

//
////////////////////////////////////////////////////////////////////////////////

////////////////////////////////////////////////////////////////////////////////
// Persistence code (featuring a pre-existing User)
//

////////////////////////////////////////////////////////////////////////////
    // Values returned by methods of non-GAE classes
    //

    CloudSession ssnSession = CloudSession.get();

    // The result has been fetched and detatched within its own transaction.
    User userLoggedOn = ssnSession.getLoggedOnUser(null);

    PersistenceManagerFactory pmf =
DataExchange.getPersistenceManagerFactory();

    //

////////////////////////////////////////////////////////////////////////////

    PersistenceManager pm = pmf.getPersistenceManager();
    Transaction tx = pm.currentTransaction();

    try
    {
      tx.begin();

   // Value returned by a method of a non-GAE class.
   // We re-fetch here to have a fresh instance fetched within this
transaction.
      userLoggedOn = DataExchange.findUserByID(tx, userLoggedOn.getID());

      Item itemCreate = new Item();
      userLoggedOn.addItem(itemCreate);

      // This appears to be optional, but do it anyway to be complete
      itemCreate.setUserParent(userLoggedOn);

      tx.commit();
    }
    finally
    {
      try
      {
        if (tx.isActive())    // Because of an exception, say
        {
          tx.rollback();
        }
      }
      finally
      {
        pm.close();
      }
    }

//
////////////////////////////////////////////////////////////////////////////////

Original issue reported on code.google.com by IanMarsh...@gmail.com on 13 Jan 2010 at 10:15

[GoogleCodeExporter]

Interesting bug, thanks for the report.  First the bad news: The datastore does 
not 
support sorting by the id component of the key (or the name component for that 
matter).  The bug is that we're not detecting that you're trying to sort by the 
id 
component of the key - we should be throwing an exception.  Instead, we just 
pass 
loID through to the query.  As for the intermittent failures, I think I know 
what's 
going on.  If your first access of the list is a call to isEmpty() or size() we 
issue a 
count() query.  A count() query ignores sort orders, so the fact that you're 
trying to 
order the list by a property that the datastore doesn't know about it doesn't 
matter 
and you get the correct answer.  However, if you then try to access an element 
of the 
list we reissue the query, this time with the requested sort order.  Since none 
of the 
entities in the datastore have a property named loID, no entities match and the 
result  
of the query is empty.  Wild.

Now the good news.  Since User is always a top-level entity for you, there 
should be 
no difference between sorting by the encoded key and sorting by the id 
component 
of that string - both orderings are stable and semi-arbitrary.  I'll get this 
fixed, but if 
you switch to sorting by encoded key (as you've already discovered) you should 
be 
fine.

Thanks,
Max

Original comment by max.r...@gmail.com on 13 Jan 2010 at 9:32

• Changed state: Accepted
• Added labels: FoundIn-1.0.4, TargetRelease-1.1.0

[GoogleCodeExporter]

Updating the issue title to better reflect the underlying problem

Original comment by max.r...@gmail.com on 13 Jan 2010 at 9:47

• Changed title: Throw exception when filtering or sorting by pk-id or pk-name fields

[GoogleCodeExporter]

Original comment by max.r...@gmail.com on 14 Jan 2010 at 5:56

• Changed state: Fixed

[GoogleCodeExporter]

Boy, that was quick Max. Thanks!

Original comment by IanMarsh...@gmail.com on 14 Jan 2010 at 7:32

[GoogleCodeExporter]

Original comment by max.r...@gmail.com on 26 Jan 2010 at 12:17

• Added labels: TargetRelease-1.0.5
• Removed labels: TargetRelease-1.1.0

[GoogleCodeExporter]

Original comment by max.r...@gmail.com on 26 May 2011 at 11:21

• Changed state: Verified