Need to support bulk loader

[GoogleCodeExporter]

Especially StructuredProperty doesn't work out of the box.

For more details see 
https://groups.google.com/group/appengine-ndb-discuss/browse_thread/thread/541e2
559bb3eaa42

esp. the solution in https://gist.github.com/1145809

Original issue reported on code.google.com by guido@google.com on 15 Aug 2011 at 3:31

[GoogleCodeExporter]

Is the bulk loader working, now that ndb is officially included in SDK 1.6.0?

Also, if we do not need to use the StructuredProperty, can we use the 
bulkloader normally, without relying on the script you posted? 
https://gist.github.com/1145809 

(Your wording here of "Especially StructuredProperty" implies that bulkloader 
should, no guarantees, roughly work otherwise?)

Thanks!

Original comment by EricHF...@gmail.com on 21 Nov 2011 at 9:02

[GoogleCodeExporter]

I'm sorry, I haven't done anything about this issue yet. I don't know the bulk 
loader very well; perhaps someone who has used it can contribute a patch? We'll 
have to get this fixed before we lift the "experimental" attribute from NDB, 
but there are other more pressing issues first.

Original comment by guido@google.com on 21 Nov 2011 at 5:50

[GoogleCodeExporter]

Original comment by guido@google.com on 8 Dec 2011 at 12:27

• Added labels: Priority-Medium
• Removed labels: Priority-Low

[GoogleCodeExporter]

Original comment by guido@google.com on 8 Dec 2011 at 12:28

• Added labels: Type-Enhancement
• Removed labels: Type-Defect

[GoogleCodeExporter]

Having experimented a bit with the bulk loader, perused its documentation, and 
read over the example at https://gist.github.com/1145809, I think this can 
mostly be dealt with through documentation (which, alas, is a general problem 
with the bulk loader). Just don't set use_model_on_export. Also note that 
there's a large section in the bulkloader docs that's deprecated: everything 
between "(Deprecated) Using Code for Configuration" and "Command Line 
Arguments".

The post_export_function and post_import_function options are useful for all 
repeated items, not just for structured properties, and this is true regardless 
of whether ext.db or NDB is used. These receive a dict and a datastore.Entity 
instance -- the latter is undocumented but essentially a subclass of dict 
representing the entity.

For now, I'm lowering the priority here, since AFAICT there's no code that 
needs to be written before the next release. Let me know if I missed something.

Original comment by guido@google.com on 14 Dec 2011 at 2:51

• Added labels: Priority-Low
• Removed labels: Priority-Medium

[GoogleCodeExporter]

thanks very much guido, from all of us who starred this issue. 

Original comment by EricHF...@gmail.com on 14 Dec 2011 at 2:55

[GoogleCodeExporter]

Oh, I forgot. If you have a non-repeated structured property, you just need to 
know how to write the property_map. Basically, for a structured property foo 
with sub-properties bar and baz, you need two entries:

    - property: foo.bar
      external_name: foo.bar
    - property: foo.baz
      external_name: foo.baz 

(and various other accoutrements depending on the type and expected values).

Original comment by guido@google.com on 14 Dec 2011 at 2:56

[GoogleCodeExporter]

Original comment by guido@google.com on 12 Jan 2012 at 7:50

• Added labels: Priority-Medium