search

neos / neos-development-collection

The unified repository containing the Neos core packages, used for Neos development.
https://www.neos.io/
GNU General Public License v3.0
264 stars 222 forks source link

NodeData // Workspace ORM Relation causes trouble #3280

Closed regniets closed 2 years ago

regniets commented 3 years ago

Description

While updating neos.io to 5.3.1 we got an issue with the marketplace:sync command* that runs painfully slow (20 packages/h, compared to 50/min on current 4.3). Digging deeper into that issue i ran into an uncatched exception. This exception leads to everything staying in memory causing the command to die in a memory_limit fatal. This issue might occur in many Neos 5.* instances unnoticedly causing trouble (if it proves to be true)
(See also https://github.com/neos/Neos.NeosIo/pull/373)

* the command is calling the packagist API and converting all neos-* packages into nodes

Steps to Reproduce

  1. Setup neos.io instance on branch update-neos5 (on localbeach)
  2. run ./flow marketplace:sync
  3. Follow the log outputs (System_Development.log)
  4. Check the occuring Exceptions stating something like Exception in line 3460 of /application/Packages/Libraries/doctrine/orm/lib/Doctrine/ORM/UnitOfWork.php: A new entity was found through the relationship 'Neos\ContentRepository\Domain\Model\NodeData#workspace' that was not configured to cascade persist operations for entity: Neos\ContentRepository\Domain\Model\Workspace@000000004511d53c0000000075f9e001. To solve this issue: Either explicitly call EntityManager#persist() on this unknown entity or configure cascade persist this association in the mapping for example @ManyToOne(..,cascade={"persist"}). If you cannot find out which entity causes the problem implement 'Neos\ContentRepository\Domain\Model\Workspace#__toString()' to get a clue.
  5. Fixing it with (cascade={"persist"}) makes the command run, but it leads to other exceptions
kdambekalns commented 3 years ago

@regniets That was definitely caused by updating 5.3.0 to 5.3.1?

regniets commented 3 years ago

Sorry - no, that was by updating from 4.3 to 5.3.1

kdambekalns commented 3 years ago

I can indeed see that happen, it occurs after the 21st package during persistAll(). The workspace that triggers this is the live workspace, I am still unable to explain what happens exactly…

Sebobo commented 3 years ago

Seems we have a similar problem now in a customer project during a node migration that creates childnodes. The first transformation with createNode works fine but the same exception occurs for the second node that is transformed.

regniets commented 3 years ago

Very interesting - @kdambekalns might have an idea what caused this issue.

kdambekalns commented 2 years ago

Coming back to this, I think the use of Storage caused that, see https://github.com/neos/Neos.NeosIo/commit/efb8b62cceca7af2937142178512e61950358c14#diff-36f8c6861843271d88ccf31feec76410e42f1b7f6e938104cff1043ac4a8c146 for the fix to that. 🤷‍♂️

sorenmalling commented 2 years ago

@kdambekalns Was this issue directly related to the Marketplace package or a general Neos CMS thing?

kdambekalns commented 2 years ago

That was happening when importing/indexing packages in the Marketplace.

sorenmalling commented 2 years ago

That was happening when importing/indexing packages in the Marketplace.

Was the solution to the issue a change in how Marketplace handles the creation or do we have a issue in CMS/Flow that we need to address in more details?

kdambekalns commented 2 years ago

I think it's not really generally an issue in Flow, see https://github.com/neos/neos-development-collection/issues/3280#issuecomment-968707828

I think the use of Storage caused that, see neos/Neos.NeosIo@efb8b62#diff-36f8c6861843271d88ccf31feec76410e42f1b7f6e938104cff1043ac4a8c146 for the fix to that. 🤷‍♂️

kdambekalns commented 2 years ago

Of course the fact that @Sebobo had a similar issue might point in the direction of a similar issue. But I think it boils down to inatcnes being kept in memory and wrecking havoc. And that is something we can hardly solve "from the outside", I fear.

sorenmalling commented 2 years ago

@kdambekalns So this issue can be closed, if I get the conslusion right? (cleaning up the project board for 7.3)