Jumoo / uSyncMigrations

Rough and ready migration code.
Mozilla Public License 2.0
46 stars 61 forks source link

LeBlender #78

Open runegronkjaer opened 1 year ago

runegronkjaer commented 1 year ago

Are there any experience with LeBlender for the Grid editor? Seems like that will be a hassle to migrate as there's no similar thing in Umbraco 10+? Maybe an upgrade to the new Block Grid is the way to go?

leekelleher commented 1 year ago

@runegronkjaer I have done a Grid+LeBlender migration to Block List editor. But I don't have any generic code to share at this time.

In my case, I'd already recreated the LeBlender editor definitions in my new Umbraco 10+ install, so my custom migration code was ultimately trying to take the JSON from Grid+LeBlender and transform it into Block List's JSON structure.

I used a custom ISyncPropertyMigrator to target the v7 Grid, similar to this one... https://github.com/Jumoo/uSyncMigrations/blob/4.0.0-beta12/MyMigrations/GridToBlockListMigrator.cs#L27

Then inside the GetContentValue() method, looped over all of the Grid cells (because they were being flatted into a single column for Block List), converting each cell JSON, creating a BlockItemData object, (then adding the UDI reference a BlockListLayoutItem object, putting that inside the containing BlockValue object, which I then serialized and returned).

It wasn't the prettiest code, but it worked for my scenario.


Aside from that, it might be worth taking a look at the built-in Grid to Block Grid migration?

runegronkjaer commented 1 year ago

Hi Lee,

Thankyou for the input. Sounds like it must have taken quite some time to do? And yes, I would have to see if the Grid to Block Grid migration can take me at least some of the way. I hoped someone could tell med that the code already did what we needed to do. At least so and so far. That said it looks like I can make a ISyncBlockMigrator to handle the LeBlender blocks.

runegronkjaer commented 1 year ago

I am actually getting somewhere with this in my own fork. Only thing I'm missing now is to make migrations create xml config files for the new document type elements I need.

I thought this method would do it: context.ContentTypes.AddElementTypes();

No new file and therefore no elementtypes are created though. Is there a way to do that? I actually need the same functionality for my Archetype to block list migrator. In that case I just created the Element types beforehand, and that worked. It's not very convenient though.

KevinJump commented 1 year ago

Hi,

yeah we do that with the grid -> block grid migrator here : https://github.com/Jumoo/uSyncMigrations/blob/main/uSync.Migrations/Migrators/BlockGrid/Config/GridToBlockGridConfigBlockHelper.cs#L28-L62

its

context.ContentTypes.AddNewContentType()

and you need to pass it a model of NewContentTypeInfo (see https://github.com/Jumoo/uSyncMigrations/blob/main/uSync.Migrations/Models/NewContentTypeInfo.cs#L3)

Its done during the config pass and it then sets it up for the ContentType Handler to create them when it runs through its part of the migtation.

runegronkjaer commented 1 year ago

Looks like the Key and Alias was taken from the "Items" part and not the attribute in the xml when creating the xml for the data-types so Guid.Empty was used for all of them and only the last one would actually get a file. After I fixed that the doc types are actually starting to be added 🥳 This is great. I think I'm hacking the system in some places to get it to work though. I'm not 100% sure of what I'm doing, just what the goal is 😊

KevinJump commented 1 year ago

This is great. I think I'm hacking the system in some places to get it to work though. I'm not 100% sure of what I'm doing, just what the goal is 😊

That's the sprit, - all hacks welcome, if the core needs to do something to make this easier, add it, and then we will merge it in to make the whole process easier for the next person who comes along 😄

runegronkjaer commented 1 year ago

Will do. I have fixed some bugs that become visible with the very old and broken data I'm working on right now. When I'm done you can have a look at it. Not used to contributing though, so I don't know how.

KevinJump commented 1 year ago

No worries,

You can submit a Pull Request (PR) and we can look at it there, but if you need a hand doing that, drop me a email (kevin@jumoo.co.uk) when you're ready and i will happily go over it all with you. 👍

runegronkjaer commented 1 year ago

I have another issue now, when migrating the content from the grid to the block grid. Looks like I actually have all my settings in order and the new content element types are are created and all that.

When starting the content migration the MigrationHandlerBase.PrepareMigrations() is run. That method loads all source files but not all my new content element types from the uSync/Migrations folder. That means the mapping of the content blocks fails because the content types are missing from the context.ContentTypes.

How would I get around that?

KevinJump commented 1 year ago

🤔 - So are you converting from leblender to grid, or doing a one step leblender to block list ?

i think the workflow that works (but doesn't have to be the one we stick too!) - is run a migratrion from v7 up. and run the import. then run a conversion to block grid.

I can see this might not be ideal if you want to do it in one step. so maybe we can workout how to fix that.

-- at the moment the reason we have conversions for block grid, is because we need to know the grid config. and the grid config comes from files outside of uSync (its principally the grid.config.js file in config but it can also be bits of package. Manifests from packages with grid config.

So to get that from a v7 site we would need some way of extracting it (i am considering a small v7 package that grabs the combined grid.config for a site. and puts it in the root of the uSync folder. then we could pick it up as part of a migration?)

runegronkjaer commented 1 year ago

I'm converting leblender grid directly to block list, and so far it's actually working just fine. I have made a few adjustments to the v8 grid to block grid migration and that was enough for the settings part.

The content "just" needs knowledge of the work that was done in the settings migration. Otherwise it does not know about the new content types I made from the leblender settings in the grid.editors.config.js.

For my Archetype migration I would actually need the same, as that migration also has to invent new content types on the fly.

Is it possible to let the content migration know about the NewContentTypeInfo's I'm creating?

KevinJump commented 1 year ago

I think the new Content types are added in memory but when the ContentType bit of the migration runs it "Should" output them along with all the content types to the ContentTypes folder of the migration output ?

As long as you are creating them before the content type migration runs (so during the config values bit of your migrator?)

runegronkjaer commented 1 year ago

Interesting thought. I did not try to run both of them in succession. I will try that monday :)

Have a great weekend.

runegronkjaer commented 1 year ago

I think the new Content types are added in memory but when the ContentType bit of the migration runs it "Should" output them along with all the content types to the ContentTypes folder of the migration output ?

As long as you are creating them before the content type migration runs (so during the config values bit of your migrator?)

Looks like the SyncMigrationContext is created from files on each migration. That means the memory from the Settings migration is overwritten with content from the files when starting the Content migration. So I'm back to square one. Could I somehow make it run the Settings migration before the Content migration, so the memory will be filled with all the generated stuff? That seems like what need to happen. Not quite sure how though.

runegronkjaer commented 1 year ago

OR, Can I make use of Umbraco's ContentType service and find them that way? They are imported and I have the alias, så all I need is the Guid. Would that be a sane way of doing it?

runegronkjaer commented 1 year ago

OR, Can I make use of Umbraco's ContentType service and find them that way? They are imported and I have the alias, så all I need is the Guid. Would that be a sane way of doing it?

I don't think this is an option. Each property needs to be migrated as well. To do that I need an EditorAliasInfo with the old and the new editor alias. That will be a problem for me to get.

KevinJump commented 1 year ago

I think the thing you may be missing as a way to reimport the same migrated files ??

so if you run a "full" (settings + content) migration against your existing files it will create all the uSync files in folder inside usync/migrations .

now in the basic setup, "do it, do it now" will run a full import against that folder, so import all the new settings, etc and all the new content".

so what you want ? (i am guessing) is to run the settings import from that folder separately from the content one ?

but - just to say the theory of migrations is you can run the full migration each time - because the outputted migration settings/content will in theory be the same each time, the actual cost of running the settings import should be low.

does that make sense, or am i missing something (which i consider to be highly likely here!)

runegronkjaer commented 1 year ago

I actually think I did it. And without using any Umbraco services. Have been cracking hard at it since yesterday. My head almost exploded but luckily it started working before that happened 😅

BPadfield commented 1 year ago

I've built my own LeBlender migrator but it currently depends on getting the datatypes using the DataTypeService which in turn means having to import all the datatypes from the version 7 migration to then do the grid to blockgrid migration. Is there a way of getting the previous migration information within the current migration i.e. within the AdditionalContentTypes method?

KevinJump commented 1 year ago

Hi,

yes although it does depend on what information you need.

we store information in the MigrationContext. - speicifcally the DataTypeMigrationContext. https://github.com/Jumoo/uSyncMigrations/blob/main/uSync.Migrations/Context/DataTypeMigrationContext.cs

so you can normally access this via context.DataTypes so for example if you want to get the alias of a DataType from the GUID value you can call context.DataTypes.GetAlias(dataTypeGuid) to get it.

if you need more information

christophborndigital commented 1 year ago

@runegronkjaer is your LeBlender migration code available anywhere? We're trying to implement this ourselves for going from Grid to BlockGrid, but not getting far yet.

We first looked at implementing the code as SyncPropertyMigratorBase based on MyMigrations/DTGEMigrator/DTGEMigrator.cs, then figured in order for it to work as a Grid editor it may need to be implemented as a ISyncBlockMigrator (similar to uSync.Migrations/Migrators/BlockGrid/BlockMigrators/DocTypeGridEditorBlockMigrator.cs)

I've tried to add my (stub) migrator in a composer with

        builder
            .WithCollectionBuilder<SyncBlockMigratorCollectionBuilder>()
                .Add(typeof(LeBlenderBlockMigrator));

but so far it's not even hitting any of the implemented methods when running the migration.

My plan was to use the grid editor or view to determine if it's a LeBlender item to then handle it.

Am I on the right track?

I also noticed that the grid.editors.config.js in the Umbraco 7 package export only had an Umbraco Forms macro in it, none of the other editors, I replaced that file manually in the zip file and tried with that, but that didn't seem to make a difference.

I may need to just plug this entire package code into our project to property understand how it all ties together.

runegronkjaer commented 1 year ago

@runegronkjaer is your LeBlender migration code available anywhere? We're trying to implement this ourselves for going from Grid to BlockGrid, but not getting far yet.

Our best (or most hacked) code is this: https://github.com/runegronkjaer/uSyncMigrations/tree/eforsyning-migration

This code is updated from the original uSyncMigrations in the first part of summer. Since then it has been changed quite a bit though. Especially changes regarding grid to blockgrid. It is working on Umbraco 7 data.

Note that to get anything working you need the /config/grid.editors.config.js from your original Umbraco 7 project.

christophborndigital commented 1 year ago

@runegronkjaer thank you for the prompt response, I will check that out