Serialization of Clousure is not allowed [v:render.uncache]

codegm commented 4 years ago

With TYPO3 v10 (current master) it throws an exception when i use the uncache ViewHelper

I think the controllerContext is the problem.

vhs: current dev Php: 7.4

NamelessCoder commented 4 years ago

Are you certain you are only passing scalar/null argument types, or plain arrays containing only scalar/null? It is almost always a problem of arguments - you have to pass the least complex variable (e.g. uid, not the entire object) to this ViewHelper.

codegm commented 4 years ago

It comes with an update from TYPO3 v9.

But i set arguments="{test: data.uid}" or arguments="" and the exception is the same.

The exception comes from: sysext/core/Classes/Cache/Frontend/VariableFrontend.php line 60

But when I do some serialize direct in the ViewHelper it throws there the exception $test = serialize($renderingContext->getControllerContext()); or $test = serialize($renderingContext->getControllerContext()->getUriBuilder());

typo3ua commented 4 years ago

I have the same problem... After update from 9.5.16 to 9.5.17,

My current example: <v:render.uncache partial="ItemLayout/Button/AddNew" arguments="{uid:'{pageData.uid}',navtitle:'{pageData.nav_title}'}"/>

I tried to use this... but without change <v:render.uncache partial="ItemLayout/Button/AddNew" arguments="{uid:'{pageData.uid}'}"/>

I get an error Cannot serialize

stack_trace.txt

System:

PHP 7.2
TYPO3 9.5.17
VHS 6.0.0

I added the new issue here https://forge.typo3.org/issues/91404

NamelessCoder commented 4 years ago

Looks like I have to see what can be done about this in the uncache ViewHelper, because the serialization behavior in TYPO3 is unlikely to change (or change enough to fix this problem without requiring any work).

Zillion01 commented 4 years ago

Same problem here on 9.5.17

<v:render.uncache partial="Topic/Reply" arguments="{topic:topic, goToPageAfterCreate:goToPageAfterCreate}" />

NamelessCoder commented 4 years ago

Could you try this with today's release of TYPO3 (all currently supported branches)? Patches have been made there to mitigate some of the regressions caused by the serialization limitations.

Zillion01 commented 4 years ago

Hey thanks, on 9.5.18 no more errors!

dev-for-web commented 4 years ago

I still get kind of the same error with 9.5.18. But this time not in the serialize method, now in the unserialize.

System:

PHP 7.2
TYPO3 9.5.18
vhs 6.0.0

Cannot unserialize TYPO3\CMS\Core\Locking\FileLockStrategy

codegm commented 4 years ago

With TYPO3 10.4.3 my problems are gone!

typo3ua commented 4 years ago

@Zillion01 I still get kind of the same error with 9.5.18. But this time not in the serialize method, now in the unserialize.

System:

PHP 7.2
TYPO3 9.5.18
VHS 6.0.0

No, it is not resolved! I had updated to 9.5.18 but I still see this error...

but not immediately ... something changed

The previous time I saw an error immediately as I go to the page where I have this element with VHS...

This time all not so... I get an error when you view the same page again

I get an error when I visit one more time the page (revisit a page) where I have this element with VHS...

Step 1:

I go to the page where I have this element with VHS - No error

Step 2:

I go to another any page where I do not have this element with VHS - No error

Step 3:

I return to the page where I have this element with VHS... which I have already visited - I get this error

Conclusion:

I get an error when I view the same page (a content element with VHS) again

After Clear cache, the error disappears...

But when I repeat steps 1 - 2 - 3 ... I get an error again

NamelessCoder commented 4 years ago

The base ViewHelper logic works and does not cause serialization problems - however, your arguments are also now subject to constraints added in TYPO3 to prevent unsafe deserialization, so you may need to thoroughly review every argument you pass to v:render.uncache to truly fix this problem.

As for the FileLockStrategy problem, this most likely exists as a depdendency (of a dependency of a dependency etc.) in one or more of the arguments passed to f:render.uncache.

Closing the issue since the base problem no longer exists.

typo3ua commented 4 years ago

....so you may need to thoroughly review every argument you pass to v:render.uncache to truly fix this problem.

Very interesting, where can I find out, what can be passed that is not?

I did it... I mean I thoroughly reviewed every argument I pass to `<v:render.uncache/>`

It is my experiment. The very simple example:

In my Layouts - List.html

<html xmlns:f="http://typo3.org/ns/TYPO3/CMS/Fluid/ViewHelpers" xmlns:n="http://typo3.org/ns/GeorgRinger/News/ViewHelpers" xmlns:v="http://typo3.org/ns/FluidTYPO3/Vhs/ViewHelpers" data-namespace-typo3-fluid="true">
<f:if condition="{news}">
<f:then>
[ ===  nothing === ]
</f:then>
</f:if>
</html>

In my Partials - AddNew.html

<html xmlns:f="http://typo3.org/ns/TYPO3/CMS/Fluid/ViewHelpers" xmlns:n="http://typo3.org/ns/GeorgRinger/News/ViewHelpers" xmlns:v="http://typo3.org/ns/FluidTYPO3/Vhs/ViewHelpers" data-namespace-typo3-fluid="true">
[ ### nothing ### ]
</html>

No nothing, no problems

Now:

In my Layouts - List.html

<html xmlns:f="http://typo3.org/ns/TYPO3/CMS/Fluid/ViewHelpers" xmlns:n="http://typo3.org/ns/GeorgRinger/News/ViewHelpers" xmlns:v="http://typo3.org/ns/FluidTYPO3/Vhs/ViewHelpers" data-namespace-typo3-fluid="true">
<f:if condition="{news}">
<f:then>
<v:render.uncache partial="ItemLayout/AddNew" arguments="{uid:'{pageData.uid}'}"/>
</f:then>
</f:if>
</html>

In my Partials - AddNew.html

<html xmlns:f="http://typo3.org/ns/TYPO3/CMS/Fluid/ViewHelpers" xmlns:n="http://typo3.org/ns/GeorgRinger/News/ViewHelpers" xmlns:v="http://typo3.org/ns/FluidTYPO3/Vhs/ViewHelpers" data-namespace-typo3-fluid="true">
[ ### uid: {uid} ### ]
</html>

I get an error

(1/1) #1588784142 BadMethodCallException
Cannot unserialize TYPO3\CMS\Core\Locking\FileLockStrategy

Then Originally posted by @typo3ua in https://github.com/FluidTYPO3/vhs/issues/1647#issuecomment-631498498

What is wrong with my code and argument?

...write where to look or what to look for?

Closing the issue since the base problem no longer exists.

I think the problem is not resolved... The v:render.uncache is blocked.

typo3ua commented 4 years ago

I seem I found the culprit... it is the ext:news

I did a new clean install - TYPO3 9.5.18 without an additional custom extension.

Than I added <v:render.uncache partial="Uncache" arguments="{uid: data.uid}"/>

The <v:render.uncache/> does work well.

Next step I added custom extension ext:bootstrap_package

I added to him <v:render.uncache partial="Uncache" arguments="{uid: data.uid, height: settings.logo.height, nav: mainnavigation.0.data.uid}"/> and checked {_all -> f:debug(title:'Bootstrap Package')}

### Bootstrap package - uncache work well -> TYPO3 9.5.18 ###
page uid: 1: | logo height: 52 | uid nav item: 4

bs_debug

The ext:bootstrap_package passed as arguments only that I wrote and a little more... just because passed argument height: settings.logo.height related to settings.logo and located inside

debug_bs

Next step I added custom extension ext:news

I added to him <v:render.uncache partial="Detail/Uncache" arguments="{page: pageData.uid, news: newsItem.uid, content: contentObjectData.uid}"/> and checked {_all -> f:debug(title:'Detail News')}

### Detail news -> uncache works well -> TYPO3 9.5.18 ###
uid page: 4 | uid news: 3 | uid content: 6

news_debug

...but when I repeat visit the detail news page I get an error

Oops, an error occurred!
Cannot unserialize TYPO3\CMS\Core\Locking\FileLockStrategy
More information regarding this error might be available online.

debug_news

The reason as I think the settings array which I do not pass as arguments and it is not under the control, it is very large... I passed only arguments="{page: pageData.uid, news: newsItem.uid, content: contentObjectData.uid}" it passes the whole array - settings array

news_debug_all

...and therefore сan not unserialize

I think there are two ways to resolve it:

Not to pass all arguments that are not specified in the arguments' attribute. It can do an authorext:news`.
Restrict the passing of all arguments that are not specified in the 'arguments' attribute. It can do an author ext:vhs and this is will better for the future.

Related links:

typo3ua commented 4 years ago

After hint #1656

Now I did so...

<v:render.uncache partial="Detail/Uncache" arguments="{page: pageData.uid, news: newsItem.uid, content: contentObjectData.uid, settings: '{startingpoint: settings.startingpoint}'}"/>

debug_vhs

I again get an error

Oops, an error occurred!
Cannot unserialize TYPO3\CMS\Core\Locking\FileLockStrategy
More information regarding this error might be available online.

dev-for-web commented 4 years ago

Same here. I can even overwrite the settings variable with an integer value and pass no other argument into the viewhelper. Still get an error:

Cannot unserialize TYPO3\CMS\Core\Locking\FileLockStrategy

typo3ua commented 4 years ago

Same here. I can even overwrite the settings variable with an integer value and pass no other argument into the viewhelper. Still get an error:

Can you test v:render.uncache without ext:news with or without other extensions?

.... and add your here... https://forge.typo3.org/issues/91451 and here https://github.com/georgringer/news/issues/1290

typo3ua commented 4 years ago

...then how to define the cause and to whom to write?

NamelessCoder commented 4 years ago

Georg would be able to modify EXT:news to not assign unserializable objects to {settings} but I honestly think that he won't implement such a change - he would likely view it as very breaking. From my own perspective I still think that it's technically wrong to place such values in {settings} since normally, this array is filled with TS and FlexForm settings which by default cannot assign unserializable objects.

I still can't be completely sure but assuming that the {settings} thing really is the only remaining problem, you might be able to work around that by controlling this variable more precisely.

Create a new partial to contain the call to v:render.uncache and then use f:render to render that partial, but limit the settings array. This will make the context inside the new partial contain a limited set of variables in {settings}, hopefully avoiding the unserializable ones.

<f:render partial="PartialWithUncachedCall" arguments="{settings: {foo: settings.foo, bar: settings.bar}}" />

Using this method you can create a custom {settings} variable that only contains the precise values that you passed. When v:render.uncache then serializes the context, the problematic variables will no longer be included in the context seen by v:render.uncache and serialization should work.

You are welcome to report this to Georg and ask for his opinion, but I have to close this discussion now and say that VHS cannot reasonably be expected to scan every single part of the context to determine if it is safe to serialize or not. I've determined that the serialization does work as long as the context does not contain unserializable objects and the rest must be left up to you as the implementer, to ensure that context won't contain such variables. This can be done with the method above, or you can create a custom ViewHelper that creates your limited {settings} variable, or you can do so in a controller action - but all of this is outside of VHS, and VHS will not be implementing any special workarounds to accommodate specific extensions - regardless of how popular they are. For the record, this same problem would exist if you passed the problematic variable as data argument for f:cObject so it is not exclusive to VHS.

typo3ua commented 4 years ago

Thanks for the detailed explanation and your try to help.

@georgringe wrote: ... https://github.com/georgringer/news/issues/1290

IMO this is a combination of security bugfix in latest core + vhs. I won't change behaviour here

... @NamelessCoder wrote: For the record, this same problem would exist if you passed the problematic variable as data argument for f:cObject

you are right... https://forge.typo3.org/issues/91451#note-4

I am now in search of the resolve... and certainly to try your tips... I will write about their results here.

typo3ua commented 4 years ago

@NamelessCoder wrote: Create a new partial to contain the call to v:render.uncache and then use f:render to render that partial, but limit the settings array. This will make the context inside the new partial contain a limited set of variables in {settings}, hopefully avoiding the unserializable ones.

I did it:

Install a clean system last version TYPO3 and ext:bootstrap_package and ext:news

1 step: In the /ext/news/Resources/Private/Templates/News/Detail.html I added

<f:render partial="Detail/RenderWithoutUncache" arguments="{
    page: pageData.uid,
    news: newsItem.uid,
    content: contentObjectData.uid,
    settings: '{startingpoint: settings.startingpoint}'
}"/>

2 step: in the /ext/news/Resources/Private/Partials/Detail/RenderWithoutUncache.html I added

{_all->f:debug(title:'Render Without Uncache Partial')}

<v:render.uncache partial="Detail/RenderWithUncache" arguments="{
    page: page,
    news: news,
    content: content,
    settings: settings
}"/>

<h3>Render Partial - "f:render"</h3>
<ul>
    <li>Page UID == {page}</li>
    <li>News UID == {news}</li>
    <li>Content UID == {content}</li>
    <li>News settings (startingpoint) == {settings.startingpoint}</li>
</ul>

3 step: In the /ext/news/Resources/Private/Partials/Detail/RenderWithUncache.html I added

{_all->f:debug(title:'Render With Uncache Partial')}

<h3>Render Partial - "v:render.uncache"</h3>
<ul>
    <li>Page UID == {page}</li>
    <li>News UID == {news}</li>
    <li>Content UID == {content}</li>
    <li>News settings (startingpoint) == {settings.startingpoint}</li>
</ul>

Results:

Debug: render_partials

Render

I go to the list news then click record of news and go to the detail news all work well I see detail news and my technical information News->List->News->detail

Render Partial - "v:render.uncache"

- Page UID == 2
- News UID == 4
- Content UID == 2
- News settings (startingpoint) == 3

Render Partial - "f:render"

- Page UID == 2
- News UID == 4
- Content UID == 2
- News settings (startingpoint) == 3

But that is where it all well ends.... when I revisit again the same page I get this error

cannot_unserialize

This error appears when I revisit again the same page where the TYPO3 first can unserialize but then cannot unserialize.

When I first-time visite the page, where the happening "unserialize" and the TYPO3 can "unserialize" something then all works well and this error does not appear.

This error appears when I revisit again the same page where the TYPO3 first can unserialize but then cannot unserialize.

Zillion01 commented 4 years ago

I seem still to get this bug

TYPO3 9.5.18, 9.5.19 and 9.5.20 VHS 6.0.3

https://forge.typo3.org/issues/91664

typo3ua commented 4 years ago

...I think that nobody does not want to solve this issue

NamelessCoder commented 4 years ago

I'm sorry you're affected by this, but I have to quote myself from earlier - and then lock this thread. This will be the final word on this matter.

I have to [...] say that VHS cannot reasonably be expected to scan every single part of the context to determine if it is safe to serialize or not. I've determined that the serialization does work as long as the context does not contain unserializable objects and the rest must be left up to you as the implementer, to ensure that context won't contain such variables.

This cannot and will not be "solved" (read: prevented) through VHS as there is no way to scan the entire context for any and all unserializable values infinitely through potentially recursive objects. You, the implementer, will have to manage your context. The problem was not introduced by VHS, it was introduced by TYPO3 deciding that certain types of objects are no longer safe to serialize and unserialize, combined with third party code assigning instances of such objects as part of the Fluid context, or implementers attempting to pass such problematic object instances as template variables. A pure TYPO3 context is known to be working, this problem is exclusively caused by third party code assigning problematic values.

There is therefore nothing I can do from the VHS perspective, and continuing to discuss it makes very little sense. I've provided you with the cause and potential solutions, now the rest is up to you (or the third party code which is assigning the problematic object instances as Fluid context variables).

FluidTYPO3 / vhs