upgrade_d7_filter_format migration causes 'Missing filter plugin: filter_null.'

[geerlingguy]

When I run drush migrate-import --group=migrate_drupal_7, I get the message:

 [error]  Missing filter plugin: filter_null. 
 1/5 [=====>----------------------]  20% [error]  Missing filter plugin: filter_null. 
 2/5 [===========>----------------]  40% [error]  Missing filter plugin: filter_null. 
 [notice] Processed 5 items (5 created, 0 updated, 0 failed, 0 ignored) - done with 'upgrade_d7_filter_format'

Might be some error with the general Drupal 7 to 8 migrate upgrade path. For now, not a major issue, but look into it later.

[geerlingguy]

It looks like 2 out of the 5 text format filters are causing this issue; my guess is it's the Markdown and Unfiltered formats (see https://www.jeffgeerling.com/admin/config/content/formats).

See related Drupal.org issue: Missing migration filters that are replaced with filter_null may have invalid settings applied.

[geerlingguy]

The temporary fix (if I can't figure out a way to make it work in automation) is to go in and re-save the filters; for the Markdown filter, I had to add the markdown filter in the settings and save it, and then blog posts rendered correctly.

[mstrelan]

From memory your Markdown text format had two filters enabled when you showed us on your D7 site. You enabled a module in order to re-save with one of those filters but ignored the other.

[geerlingguy]

Yeah. I think the other filter was for 'unfiltered'. I'll have to see if I actually have any content or blocks using that filter, and if so either deal with them (convert to a different format) or figure out some other way of handling the migration. Or just re-save the input format after the final migration (but I hate manual steps!).

[mstrelan]

I think you're mixing up filters and formats. The Markdown format had a code filter enabled in D7. I think that could be what the migration is complaining about.

[mstrelan]

See also https://www.drupal.org/project/drupal/issues/3061571

The latest patch even specifically handles codefilter and markdown. If the filters are not available on the destination site they will be omitted (rather than mapped to filter_null) and a message will be shown suggesting that you install a replacement.

So the options I see are:

• Run the migration with the patch above, install a replacement code filter later
• Allow it to migrate as filter_null, resave your text formats to clear filter_null, add the code filter later
• Install the dev snapshot of codefilter and hope for the best
• Remove codefilter from the source site prior to migrating, add the code filter later

[geerlingguy]

I might end up going with:

Remove codefilter from the source site prior to migrating, add the code filter later

I'm meaning to replace codefilter anyways, as it doesn't (and probably never will) have dark mode support, nor a D8 stable release. See https://github.com/geerlingguy/jeffgeerling-com/issues/4

I would have to change a good deal of content though—anywhere there's a <?php ?> snippet at minimum.

[mstrelan]

Yeah I guess if you use the patch then you don't need to break your Drupal 7 site and your Drupal 8 will have the same result. But then you have to manage patches, at least until the migration is complete.

On Fri, 13 Mar. 2020, 8:31 am Jeff Geerling, notifications@github.com wrote:

I might end up going with:

Remove codefilter from the source site prior to migrating, add the code filter later

I'm meaning to replace codefilter anyways, as it doesn't (and probably never will) have dark mode support, nor a D8 stable release. See #4 https://github.com/geerlingguy/jeffgeerling-com/issues/4

I would have to change a good deal of content though—anywhere there's a <?php ?> snippet at minimum.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/geerlingguy/jeffgeerling-com/issues/7#issuecomment-598460885, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAFCMFX56VUCRZFS7Z66DK3RHFPC5ANCNFSM4KXINYOA .

[geerlingguy]

In #4 I decided to just stick with Code Filter (for now, at least); it is working and has a stable-ish dev release. Though hasn't been worked on in a couple years. I believe drupal.org still uses it, though have no clue if they have a timeline for a D8 upgrade (I believe they're still on D7)...

[geerlingguy]

I'm testing a reinstall with the code from #31 to see if these messages are gone now that I have the code filter filter installed...

[geerlingguy]

Got rid of two of the warnings, but still getting:

 2/5 [===========>----------------]  40% [error]  Missing filter plugin: filter_null. 
 [notice] Processed 5 items (5 created, 0 updated, 0 failed, 0 ignored) - done with 'upgrade_d7_filter_format'

And if I look at the migrate_message_upgrade_d7_filter_format table in the D8 database, I see one row with the message:

Filter filter_markdown could not be mapped to an existing filter plugin; defaulting to filter_null and dropping all settings. Either redo the migration with the module installed that provides an equivalent filter, or modify the text format after the migration to remove this filter if it is no longer necessary.

If I look at the markdown module's code, it looks like it's doing a transformation here:

/**
 * Implements hook_migration_plugins_alter().
 */
function markdown_migration_plugins_alter(array &$migrations) {
  if (isset($migrations['d7_filter_format'])) {
    $migration = &$migrations['d7_filter_format'];

    // Add mapping from filter_markdown to markdown for D7 migratins.
    $migration['process']['filters']['process']['id']['map']['filter_markdown'] = 'markdown';
  }
}

But for some reason the migration is not picking that up... Grr.

[geerlingguy]

No clue what's going on here. I dumped the $migration structure to stdout, confirmed that the map ($migration['process']['filters']['process']['id']['map']) is using the correct structure), and re-confirmed the markdown module is installed.

[geerlingguy]

I also asked in the #migrate slack channel in Drupal slack, no response there as of yet. Scratching my head on this one. Will have to shelve it for now. But I'll still go ahead and merge what I have so far. At least we have one filter_null instead of three now :/

[mdlutz24]

I responded in slack too, but I'll add it here. When you export the migrations with migrate_upgrade, you change the migration id to upgrade_d7_filter_format, and the markdown plugin alter is still altering the original d7_filter_format, so it's not applying the alter to your migration. If you can add a custom alter that duplicates the markdown alter, except alters upgrade_d7_filter_format I expect it would work, or you could alter your migration config and add the mapping in there directly.

[geerlingguy]

@mdlutz24 - Thanks so much! I'll test this out and see how it works tomorrow.

[geerlingguy]

(By tomorrow I meant one week later, which is today :D)

[geerlingguy]

I left a new comment on Declare markdown module's Drupal 8 upgrade status—what @mdlutz24 said is correct, but I had to dig around to find that the migration key I had to override (to get the mapping actually working) was migration_config_deriver:upgrade_d7_filter_format.

[geerlingguy]

And yay, it's working!

 [notice] Processed 1 item (1 created, 0 updated, 0 failed, 0 ignored) - done with 'upgrade_block_content_entity_display'
 [notice] Processed 1 item (1 created, 0 updated, 0 failed, 0 ignored) - done with 'upgrade_block_content_entity_form_display'
 [notice] Processed 5 items (5 created, 0 updated, 0 failed, 0 ignored) - done with 'upgrade_d7_filter_format'
 [notice] Processed 3 items (3 created, 0 updated, 0 failed, 0 ignored) - done with 'upgrade_d7_custom_block'