MongoDB: Various fixes and improvements

[amotl]

About

The venerable migr8 program did not receive much love to get the details right. This patch intends to catch up on this, in order to make ctk load table mongodb://... hold up to its promise of better DWIM.

Preview

pip install --upgrade 'cratedb-toolkit[mongodb] @ git+https://github.com/crate/cratedb-toolkit.git@mongodb-better-1'

What's Inside

Bugfixes

• [x] GH-210
• [x] Fix timestamp parsing on migr8 export. Exception: Can't parse datetime string 2022-06-11T04:37:53.200Z
• [x] Converge _id input field to id column instead of dropping it. GH-215

Details

• MongoDB: Fix missing output on STDOUT for migr8 export
• MongoDB: Improve timestamp parsing by using python-dateutil
• MongoDB: Converge _id input field to id column instead of dropping it
• MongoDB: Make user interface use stderr, so stdout is for data only
• MongoDB: Make migr8 extract write to stdout by default
• MongoDB: Make migr8 translate read from stdin by default
• MongoDB: Improve user interface messages
• MongoDB: Strip single leading underscore character from all top-level fields
• MongoDB: Map OID types to CrateDB TEXT columns
• MongoDB: Make migr8 extract and migr8 export accept the --limit option
• MongoDB: Fix indentation in prettified SQL output of migr8 translate
• MongoDB: Add capability to give type hints and add transformations using --transformation, see Zyp Transformations.

[amotl]

Hi Hernan. Thanks for your suggestion. I was sure there are anomalies, but I can't spot them, probably because of operational blindness. Can I humbly ask you to submit a corresponding suggestion how and where to improve the documentation, either by commenting on this patch, or by submitting a separate one? Thanks a stack!

[amotl]

The MongoDB Table Loader will now also accept Zyp transformations for re-shaping data.

• https://cratedb-toolkit--216.org.readthedocs.build/io/mongodb/loader.html#using-zyp-transformations