Request for supporting datastore.AllocateIDRange() like Java or Python

[avvmoto]

Request for supporting datastore.AllocateIDRange() like Java or Python

Hello. Thankyou for the great services. I have a feature request today.

Feature Request

I want datastore.AllocateIDRange() to be supported, which allocate ID range specified start and end by user.

In Java or Python's datastore libraries, this feature is supported, but not in golang one. To support this fuction in golang is not only useful but also natual. e.g.

• Java
  • allocateIdRange
  • https://cloud.google.com/appengine/docs/standard/java/javadoc/com/google/appengine/api/datastore/DatastoreService.html#allocateIdRange-com.google.appengine.api.datastore.KeyRange-
• Python
  • allocate_id_range (model, start, end, **kwargs)
  • https://cloud.google.com/appengine/docs/standard/python/datastore/functions

This allocate fuction is quite useful when users have id ranges which they want to be allocated. There is similar function datastore.AllocateIDs, in which user can specify only size to be allocated, not range. So this function is not enough for this purpose.

Usecase

datastore.AllocateIDRange() is useful when users have id ranges to be allocated. For example, I want to migrate MySQL rows to Datastore Entity.

Requirements in this migration is :

• Use string Key for Entity. The value of the string key is obfuscated MySQL int ID.
• After migrate finished, I want to use the Datastore automatic allocator for new entity's key.
• the Datastore automatic allocator shold avoid the range which MySQL have already been used as IDs.

In this case, I have the range not to be allocated by the Datastore automatic ID allocator. This range is used by MySQL, and it starts with 1 and ends with the number of rows in MySQL.

How to implement

Implementation of datastore.AllocateIDRange is quite simple. Just define a new function to specify Max parameter in AllocateIdsRequest. e.g.

• Java Implementation
  • https://github.com/greghaskins/googleappengine/blob/e78ea5c1ef55564b776ff155630202afb2d4ecfa/java/src/main/com/google/appengine/api/datastore/DatastoreServiceImpl.java#L164
• Phython Implementation
  • https://github.com/optimizely/python-appengine/blob/master/google/appengine/ext/db/__init__.py#L1683

Question

Are there reasons datastore.AllocateIDRange is not supported in the golang library? For example, is there a plan to stop supporting Max Parameter at the Datastore API? If there is no problem to support it, I would like to send a PR to implement it.

[karupanerura]

How is the progress?

[leighmcculloch]

This is also necessary when copying entities from one kind to another, or from one project to another. As described in the Datastore docs:

System-allocated ID values are guaranteed unique to the entity group. If you copy an entity from one entity group or namespace to another and wish to preserve the ID part of the key, be sure to allocate the ID first to prevent Cloud Datastore from selecting that ID for a future assignment.

https://cloud.google.com/appengine/docs/standard/python/datastore/entities

There's no way to allocate a specific ID, or a range of IDs with the Go appengine/datastore package, which means there's no way to safely use auto-allocated IDs, and move entities to different kinds or different projects without risking the allocating regenerating an existing ID.