search

spicywebau / craft-neo

A Matrix-like field type for Craft CMS that uses existing fields
Other
403 stars 63 forks source link

Blocks lost when provisional draft is applied #720

Closed ccchapman closed 1 year ago

ccchapman commented 1 year ago

Bug Description

We are experiencing content loss on the latest version of all packages. The problem is intermittent and affects only some entries. We discovered the Craft CLI resave action is a temporary band-aid however the issue will come back.

ℹ️ Note: The issue can be reproduced with Typed Link Field enabled or disabled. Unsure if this makes Typed Link Field is irrelevant.

Potentially related issues:

I will email a video, DB, and composer.lock.

Thanks! 😄

Steps to reproduce

  1. Edit the entry admin/entries/reusableContent/403150
  2. Scroll to bottom of entry
  3. Change some text
  4. Wait for provisional draft to save
  5. CMD + S

Expected behaviour

Save w/o blocks going missing

Neo version

3.7.0

Craft CMS version

4.4.1

What is the affected Neo field's propagation method?

Field propagation method is all, section propagation method is siteGroup

Does this issue involve templating, and if so, is eager-loading used?

This is not a templating issue

mfell commented 1 year ago

Thanks for your bug report ... I had a similar effect, after updating Neo to 3.7.0.

It accoured under Craft CMS 4.3.10 (or 11) ... unfortunaly I had no idea what to do and because of this I keep staying on Neo 3.6.4

If needed: I can email a video, DB, and composer.lock, too.

ttempleton commented 1 year ago

Thanks for sending that email @ccchapman. Could you please also send in the composer.json and I'll have a look. Unfortunately I'm unable to reproduce this on my usual local testing environment... and also pretty curious about it seemingly only affecting 3.7.0. I can't think of any change in 3.7.0 that would cause this, but could definitely be wrong about that.

ccchapman commented 1 year ago

@ttempleton,

Thanks for your reply.

ccchapman commented 1 year ago

https://github.com/craftcms/cms/issues/12858 looks possibly related? I can try to test that tomorrow.

ccchapman commented 1 year ago

https://github.com/craftcms/cms/commit/031bfe812140c297f4d41bc2a4e99fb355cdf6ce does not address this issue.

mfell commented 1 year ago

Hi ... I just checked this issue with Craft 4.4.1 and Neo 3.7.1 -> blocks are lost.

I'm unsure, how to support your work, but I can send you my composer.json, DB, Server information ... whatever you need.

With Neo 3.6.4 -> everything's fine ... in my case.

Bildschirmfoto 2023-03-15 um 11 35 08

ttempleton commented 1 year ago

Thanks for sending that in @ccchapman. It looks like all of the blocks that disappear have their primaryOwnerId set to a provisional draft, which is pretty strange since the primaryOwnerId-setting should work the same as with Matrix blocks, and I haven't heard of any such bug affecting Matrix blocks. I'm going to try different actions to trigger the initial incorrect setting, and to figure out how best to set already affected blocks to the correct primaryOwnerId. Will update here when I have more info.

@mfell if it's only occurring for you with 3.7.0+ then I'm wondering if you have a different issue. Could you please send your composer.json/lock files and composer backup to plugins@spicyweb.com.au and we'll have a look at it.

mfell commented 1 year ago

Hi again, I'll send @ttempleton the needed files in a few minutes, a few words here:

Craft 4.4.2, Neo 3.7.1 (Update from 3.6.4)

The child-entries are lost (existing entries) Bildschirmfoto 2023-03-15 um 20 47 00

The relation parent / child of new entries is lost -> correct: has to be nested Bildschirmfoto 2023-03-15 um 20 45 22

Some server config (ddev) Bildschirmfoto 2023-03-15 um 21 13 28

sfsmfc commented 1 year ago

We have a similar error. After provisional draft is finished, we have wrong child elements. In some cases, the "new wrong" element has already content. If we store the whole entry, we can see the right elements.

This happens, if we use Neo in Version 3.7.x. If we downgrade to version 3.6.4, all is fine.

@ttempleton If you need more informations, please let me know.

vebjorn commented 1 year ago

We have a similar error. After provisional draft is finished, we have wrong child elements. In some cases, the "new wrong" element has already content. If we store the whole entry, we can see the right elements.

I have the exact same issue. To get the right content we don't have to save the entry though. Reloading the page, which then will pull in the draft fresh, will give the correct data. Almost seems like a JS-refresh-bug or something

ttempleton commented 1 year ago

Thanks for sending those files @mfell. I was able to reproduce the second part of your report where the automatically-created child blocks were being created at the wrong level, and that should be fixed now in Neo 3.7.2. I was unable to reproduce the block loss from your first screenshot - is there any action in particular that causes it for you?

@vebjorn @sfsmfc that sounds like you are being affected by #721.

mfell commented 1 year ago

@ttempleton great, thank you so much!

I think this might be the same cause ... let me check this with Neo 3.7.2, I'll send you feedback soon as possible. Today I'm out of office, but during the weekend I'll do it.

mfell commented 1 year ago

Ok, hi @ all :)

Just checked this with Craft 4.4.3, Neo 3.7.3 under MacOS 12.6.x and Chrome:

After creating a new element (1):

After page reload (2):

(1) Bildschirmfoto 2023-03-18 um 12 15 37

(2) Bildschirmfoto 2023-03-18 um 12 17 24

mfell commented 1 year ago

Hi again,

just popped up in my mind: I didn't cleaned the browser cache yesterday.

I just checked this issue again with Craft 4.4.3, Neo 3.7.3 under MacOS 12.6.x, Chrome AND deleted browser cache: It seems to be ok now ... at least in my case.

Please: May anyone confirm this?

ccchapman commented 1 year ago

I confirmed the issue described in the original comment persists on Neo 3.7.3.

Composer info 
craftcms/aws-s3                            2.0.3            Amazon S3 integration for Craft CMS
craftcms/cms                               4.4.3            Craft CMS
craftcms/feed-me                           5.1.1            Import content from XML, RSS, CSV or JSON feeds into entries, categories, Craft Commerce products, and more.
craftcms/redactor                          3.0.4            Edit rich text content in Craft CMS using Redactor by Imperavi.
diginov/craft-sentry-logger                4.1.1            Pushes Craft CMS logs to Sentry through a real Yii 2 log target
hybridinteractive/craft-position-fieldtype v4.x-dev f9c0ef9 Brings back the Position fieldtype from Craft 2
korcontrol/craft-classy                    2.0.0            Twig helpers inspired by https://github.com/JedWatson/classnames
mutation/translate                         3.0.1            Translate messages in the control panel
nystudio107/craft-closure                  1.0.0            Allows you to use arrow function closures in Twig
nystudio107/craft-retour                   4.1.11           Retour allows you to intelligently redirect legacy URLs, so that you don't lose SEO value when rebuilding & restructuring ...
nystudio107/craft-seomatic                 4.0.22           SEOmatic facilitates modern SEO best practices & implementation for Craft CMS 4. It is a turnkey SEO system that is compre...
nystudio107/craft-vite                     4.0.5            Allows the use of the Vite.js next generation frontend tooling with Craft CMS
putyourlightson/craft-blitz                4.4.3            Intelligent static page caching for creating lightning-fast sites.
sebastianlenz/linkfield                    2.1.5            A Craft field type for selecting links
spacecatninja/imager-x                     4.1.11           Ninja powered image transforms.
spicyweb/craft-embedded-assets             3.1.1            Manage YouTube videos, Instagram photos, Twitter posts and more as first class assets
spicyweb/craft-neo                         3.7.3            A Matrix-like field type that uses existing fields
topshelfcraft/environment-label            4.0.2            ...so you don't forget where you are.
topshelfcraft/wordsmith                    4.2.0            ...because you have the best words.
verbb/navigation                           2.0.16           Create navigation menus for your site.
vlucas/phpdotenv                           v5.5.0           Loads environment variables from `.env` to `getenv()`, `$_ENV` and `$_SERVER` automagically.
yiisoft/yii2-redis                         2.0.18           Redis Cache, Session and ActiveRecord for the Yii framework
yiisoft/yii2-shell                         2.0.5            The interactive shell extension for Yii framework
ttempleton commented 1 year ago

@mfell when that was happening, was there anything in particular you were editing that could have led to that error, or did it just always happen no matter what you changed on the entry? I was never able to reproduce that part of your report, so maybe it was just some sort of caching issue if clearing the cache has fixed it for you.

mfell commented 1 year ago

@ttempleton It happens always on save ... I didn't notice any special action witch makes this effect accouring.

And in my case, it's working well now ... I can't report any problem for now. (Thanks a lot!)

ccchapman commented 1 year ago

This issue has now occurred on another client. Their frontend pages were missing content. When those pages were then edited in Craft following the steps here, blocks would disappear.

This project (also multi-site) is experiencing the same behaviours as reported in original comment.

We confirmed that the affected Neo blocks had a primaryOwnerId of a provisional draft. craft entries/resave will fix the primaryOwnerId to be the currently published entry ID.

We are still unclear what causes the primaryOwnerId to be set incorrectly in the first place.

We have tested with a "good" version of the DB to check if GC triggers the issue; however, it does not appear relevant.

The provisional draft which was set as the primaryOwnerId exists in the "good" DB too. Therefore I do not believe the creation of the provisional is triggering the bad assignment?

This project has different package versions:

I can provide more (incl. "good" vs. "bad" DB) as required.

ttempleton commented 1 year ago

@ccchapman Does that project use any Matrix fields? If so, do any Matrix blocks have an incorrect primaryOwnerId? I haven't been able to reproduce the incorrect primaryOwnerId on projects other than what had already occurred on the database backup you sent in, and I can't find any difference in Neo vs. Matrix field code that would cause Neo and not Matrix to have the incorrect primaryOwnerId set. Note that Matrix fields would be unlikely to suffer block loss in this case, as it seems to be the Neo block structure saving in particular that loses the Neo blocks, due to the structure saving not being given the data for the blocks with the incorrect primaryOwnerId.

mfell commented 1 year ago

Hi, I have no idea whether this comment is helpful or not ... but this might be a JS topic and during my redactorial editing with Neo under my local dev environment, I got this (screenshot).

If this post is not useful, please feel free to ignore or delete it. :)

Bildschirmfoto 2023-03-22 um 11 34 35

ccchapman commented 1 year ago

@ccchapman Does that project use any Matrix fields? If so, do any Matrix blocks have an incorrect primaryOwnerId? I haven't been able to reproduce the incorrect primaryOwnerId on projects other than what had already occurred on the database backup you sent in, and I can't find any difference in Neo vs. Matrix field code that would cause Neo and not Matrix to have the incorrect primaryOwnerId set. Note that Matrix fields would be unlikely to suffer block loss in this case, as it seems to be the Neo block structure saving in particular that loses the Neo blocks, due to the structure saving not being given the data for the blocks with the incorrect primaryOwnerId.

Yes, the project does have entries with Matrix fields. I had a look and all of those Matrix Blocks have a primaryOwnerId of the current entry ID as expected.

A couple other facts I found today:

ttempleton commented 1 year ago

I'm still unable to reproduce it. Are there any clues you can find as to what operations were being performed by the Craft project around that minute span, that could have led to the Neo blocks being saved with the wrong primaryOwnerId?

ccchapman commented 1 year ago

I'm still unable to reproduce it. Are there any clues you can find as to what operations were being performed by the Craft project around that minute span, that could have led to the Neo blocks being saved with the wrong primaryOwnerId?

I have reviewed all available logs and did not find anything suspicious around that minute span. In fact the only activity then is the Craft session-info requests.

This makes me wonder if some kind of queue job (e.g. resave, propagation, saving Neo block structures for duplicated elements, etc.) happened around that time and triggered the issue. Unfortunately we do not have the stdout for the queue workers currently writing to a log. I do not have a way to know exactly which jobs would have been running at this time.

ttempleton commented 1 year ago

This turned out to be the Neo equivalent to https://github.com/craftcms/cms/issues/13155. I've applied the fix for https://github.com/craftcms/cms/issues/13181 to Neo, and have released Neo 3.7.9 with that fix.