Migration of custom types

[thet]

Regarding schema extenders: wouldn't it possible to even do migrations for schema extended types and other arbitrary types? I know, I always doubted that, but now I rethought that... We can of course get the extended schema with it's fields, properties and widgets and create just a new FTI type with a custom schema for that, as long as those schemata don't use custom fields and widgets, where we don't have an equivalent.

That's a shout in the blue - I don't know exactly how much effort this would be... But given a complete Dexterity API, this task sounds like it could be done in a few sprinting days.

[thet]

@davisagli - I posted that originally at https://github.com/plone/Products.CMFPlone/issues/184 but moved it over here.

[pbauer]

I really like the idea of having something to help with the migration. A full automatic-migration including creating and registering types is maybe to much.

As a first step we could allow mapping AT- to DX-Types and then AT-schema-fields (regardless if they are from a extender or a real field) to DX-schema-fields. With this information we could run a automatic migration. This way admins will szill have to model their new types in DX but won't have to write custom migrations (which is much harder than creating types).

[thet]

On Fri, 2014-02-28 at 01:23 -0800, Philip Bauer wrote:

I really like the idea of having something to help with the migration. A full automatic-migration including creating and registering types is maybe to much.

As a first step we could allow mapping AT- to DX-Types and then AT-schema-fields (regardless if they are from a extender or a real field) to DX-schema-fields. With this information we could run a automatic migration. This way admins will szill have to model their new types in DX but won't have to write custom migrations (which is much harder than creating types).

that sounds good. this way, it's also clear that the responsibility is more on the admin than us :)

i'd also love to have that. i expect that would help a lot with migrations, which otherwise are hard to sell because there is too much effort involved.

[pbauer]

Great, let's put in on the agenda for the wine-and-beer-sprint

[davisagli]

Yes, should not be too hard to do at least in the cases where the field/widget is recognized.

[thet]

i've recently done a bigger migration with a lot of custom types and schema extended ones. i'd love to discuss what i've done there at the sprint. my conclusion is, writing a custom migrator is no problem.

in short:

some useful and necessary additions in p.a.c: https://github.com/plone/plone.app.contenttypes/commits/thet (to be pull-requested). e.g. i've integrated an ICustomMigrator adapter interface. this one should then be called for every content type (e.g. when a schema extender is used for many content types, and you don't exactly know which ones because you provide that adapter within a migration script of the schema extender addon-package).

custom content type migration for collective.folderishtypes: https://github.com/collective/collective.folderishtypes/blob/dxmigration/collective/folderishtypes/dx/migration/__init__.py

schemaextended type migration for collective.richdescription (where the ICustomMigrator is used): https://github.com/collective/collective.richdescription/blob/dxmigration/collective/richdescription/dx/migration/__init__.py

[timitos]

This is a small blueprint which we discussed at the wine and beer sprint:

Goal

Provide a migration interfaces for custom/extended contenttypes from archetypes to dexterity.

Requirements

The dexterity contenttype already must be implemented and available.

Thoughts about the implementation

• We need a browserview that provides a list of all custom/extended archetype contenttypes
• For every archetyp contenttype the dexterity contenttype can be selected it shall be migrated to
• The dexterity type selection must provide an option 'Do not migrate this field' as default value.
• The view shows for every archetype contenttype a list of all defined fields.
• To every field of a archetype contenttype a field of the dexterity type can be assigned.
• The field selection must provide an option 'Do not migrate this field'.
• After entering the migration configuration in the view the data can be exported to a file.
• The entered migration configuration will be saved to an annotation.
• The view provides a possibility to load a saved configuration
• The view provides a possibility to upload a previously exported configuration
• The view should provide a possibility to start a migration for a single contenttypes
• The view should provide a possibility to start a migration of all selected contenttypes

Migration Process

1. The user starts the migration
2. The migration configuration will be saved to an annotation
3. Commit
4. Start of Migration in a new Transaction (Maybe one Transaction per Contenttype)
5. If there is an error at the migration -> Rollback Transaction
6. If migration is successfull -> Commit
7. Safe new state of migrated contenttypes in annotation
8. Commit
9. Show updated view.

[timitos]

This is a quick and dirty mockup of the migration view

[pbauer]

This is the revised mockup for the migration-form done by Vilmos Somogyi:

[pbauer]

Work on this is done in the branch issue133_custom_types_migration: https://github.com/plone/plone.app.contenttypes/compare/issue133_custom_types_migration