Migration from Translatable - duplicate page creation

[muppsy007]

We are upgrading an inherited site from 3.x to 4.x and the old site used Transatable. So we are migrating to Fluent. The site uses English and Chinese only.

I'm not sure if if this is a normal thing for Translatable, or something previous developers did, but in the old version of the site with Translatable, the site tree in the CMS simply filtered based on the selected locale. More importantly, each page has a unique URL.

E.G. English home = /home while Chinese home = /home-zh-ch

What we have found is that after running the migration, we get two sets of pages showing up in the Tree. Rather than one set that simply changes when the CMS user changes the locale. I am assuming this is all to do with the original Translatable setup have two sets of URLs and the Fluent migration treating them as separate pages. Since when we manually create a page (with fluent installed), both languages have the same slug. (/home)

Does this sound correct? And is the above setup (two urls) normal for Translatable? If so, is something missing from the Fluent migration to handle this? Any suggestions pre-migration that might resolved this?

[robbieaverill]

Hi @muppsy007, yes that's correct. Translatable created separate SiteTree records, whereas Fluent doesn't do that.

The data structures are different, so we can only make best efforts to help with migration. If Translatable has a structured relationship between the original record and the translated versions of it then we could potentially map that into localisations for a single SiteTree record with Fluent.

Perhaps you could provide a small database extract to help with that?

[rvowles]

Ok, I hit this as well - I now have two copies of each page. E.g there was a HomePage instance for English and one for Chinese. They were linked on the Translations page, so they are definitely clearly linked and both of the same type. Its almost every page that has this problem.

Unfortunately I can't have this site having massive downtime so it looks like I will have to do some custom migrations.

[wernerkrauss]

bump - any news on this? It seems, ConvertTranslatableTask is simply ignoring the old _translationgroups table?

@rvowles how did you migrate that site?

[rvowles]

It got stopped as we ran into trouble with widgets, thats has been worked around but the customer lost impetus and has had other priorities.

On Wed, Oct 3, 2018 at 5:26 AM wernerkrauss notifications@github.com wrote:

bump - any news on this? It seems, ConvertTranslatableTask is simply ignoring the old _translationgroups table?

@rvowles https://github.com/rvowles how did you migrate that site?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/tractorcow-farm/silverstripe-fluent/issues/390#issuecomment-426339464, or mute the thread https://github.com/notifications/unsubscribe-auth/AADZ2HF09lyQo9r3Z6FjM1MtDDkAkqlGks5ug5PJgaJpZM4TJSE7 .

--

Richard Vowles, Full stack - from Kubernetes, through Node & Java, Web and Mobile development in Flutter - software developer for hire!

ph: +64275467747, web: www.google.com/+RichardVowles

[oddnoc]

I'm working on updating ConvertTranslatableTask with the approach of getting all the translation groups from the *_translationgroups tables and processing each one. However, I don't see a way to set the RecordID for the converted record. My approach is to choose one record to be the default record, and use its ID as the RecordID for that translation group. This would (I think) avoid the duplicate entries in the site tree, for example.

@robbieaverill any thoughts?

[robbieaverill]

Yep that should work. Much like Versioned, the RecordID should map to the original SiteTree record (I think)

[wernerkrauss]

@oddnoc I already have the task upgraded locally (incl. unit testing), got stuck with the "old record ID" issue. Maybe we can collaborate?

[oddnoc]

@wernerkrauss Have a look at my work in progress at https://github.com/tractorcow-farm/silverstripe-fluent/compare/master...oddnoc:convert-translatable

[roshnimichael]

Hi used this branch to fix the duplicate issue master...oddnoc:convert-translatable, but after running the task all the pages get archived rather than getting published. Any support is appreciated.

[taoceanz]

I'm on the same project @rvowles was on i.e. I've also experienced the duplicated pages after running the current migration task. I tried the branch from @oddnoc which definitely merged pages so was an improvement over the original migration task, though wasn't migrating page fields correctly on Page and sub-classes of Page as it doesn't address tables for those classes, only SiteTree.

I've taken oddnoc:convert-translatable as a base and created this task. It's been tested with 2 locale only so not sure how well it will stand with 3 or more locale, but in my tests, all pages and fields migrate successfully on all classes that subclass SiteTree, so long as a SiteTree_translationgroups table exists containing data as the Translatable module would write to it. This task doesn't deal with migration of classes directly sub-classing DataObject.

~One issue with this task occurs while attempting to sync version tables. It would run into a duplicate key issue and the task would fail. If you use my task, be aware it doesn't sync history, so after migration all history tabs will appear empty or have just one publish.~ Edit: DataObject, and particularly page history migration functionality has been added. If there are any records in the _Localised_Versions tables that contain duplicate ID + RecordID + Locale compound key, they'll get skipped, so it's not a 100% migration rate maybe, but it does migrate a large portion of the Page history from SS3+Translatable to SS4+Fluent.

https://github.com/taoceanz/silverstripe-fluent/blob/update/convert-translatable-task/merge-translated-pages-and-sync-translated-fields/src/Task/ConvertTranslatableTask.php

Comparison to current Fluent master convert task: https://github.com/tractorcow-farm/silverstripe-fluent/compare/master...taoceanz:update/convert-translatable-task/merge-translated-pages-and-sync-translated-fields

[David-Malesevic]

The MigrationTask seems not to work. I have a multilingual SilverStripe 3.x site with 3 locales. In the SilverStripe 3.x the pages and DataObjects are translated with Translatable. The _translationgroups Tables and the locale of the object is stored correctly in the database.

After execution of the task in the SilverStripe 4.x the page appears 3 times in each locale in the CMS and the DataObjects are still assigned to the pages and not linked to each other. So to speak the task does nothing. It takes over the page structure from the 3.x SilverStripe installation. The _localised table is created, there are also entries, but the ID's are not set correctly.

Switzerland is a multilingual Country, we have many websites which are multilingual. Therefore it is very important that this task works and does what it should.

Unfortunately the adaptation of taoceanz (https://github.com/taoceanz/silverstripe-fluent/blob/update/convert-translatable-task/merge-translated-pages-and-sync-translated-fields/src/Task/ConvertTranslatableTask.php) did not work.

Could you please help me there?

[taoceanz]

@Exeldur what about my adaption didn't work? Had you created all your Locale with Fluent in your SS4 site before running the migration task?