Bad L parameter

[Guigue79]

Hi When users or bots try to set Language variable to fr I have this error Thu, 12 Oct 2017 09:46:25 +0200 [ERROR] request="eb2a025b3e3ec" component="DmitryDulepov.Realurl.Encoder.UrlEncoder": Bad "L" parameter ("fr") was detected by realurl...... Logs are very very very large and saturate my disk I try to ignore this param with config.linkVars = L(0-1) but it does not work

[osahner]

Same here. Logging seems to be messed up this release. One Line has more than 3mb!

[responseinformationsdesign]

RealURL also seems to ignore [SYS][systemLogLevel].

Errors get written in TYPO3-Log even [SYS][systemLogLevel] was set to only log fatal errors.

[hannesbochmann]

@responseinformationsdesign realurl uses the logging system ($TYPO3_CONF_VARS['LOG']) of TYPO3 not the simple system log. System log level has no effect on the logging system. You would need to remove the default FileWriter for log level WARNING after you copied it to the log configuration for log level ERROR. This way you should only have errors in typo3temp/logs/typo3.log. By the way this is no recommended. Warnings shouldn't be ignored. ;-)

[bavarianbytes]

Same here

[fazzyx]

We have the same situation after upgrading realurl to 2.3. TYPO3 7.6

[Batman777]

Typo3 - 6.31 Same problem with huge log files. Will there be a fix in the near future?

You would need to remove the default FileWriter for log level WARNING after you copied it to the log configuration for log level ERROR.

I did not get that. Could you please try in other words?

[hannesbochmann]

@Batman777 Put that in your AdditionalConfiguration.php:

$GLOBALS['TYPO3_CONF_VARS']['LOG']['writerConfiguration'][\TYPO3\CMS\Core\Log\LogLevel::ERROR]['TYPO3\\CMS\\Core\\Log\\Writer\\FileWriter'] = $GLOBALS['TYPO3_CONF_VARS']['LOG']['writerConfiguration'][\TYPO3\CMS\Core\Log\LogLevel::WARNING]['TYPO3\\CMS\\Core\\Log\\Writer\\FileWriter'];
unset($GLOBALS['TYPO3_CONF_VARS']['LOG']['writerConfiguration'][\TYPO3\CMS\Core\Log\LogLevel::WARNING]['TYPO3\\CMS\\Core\\Log\\Writer\\FileWriter']);

[Batman777]

Awesome .. thank you. Hope this works! :-)

[Batman777]

This morning the site was online. This afternoon i got a white page.

Log file gives me the following error: PHP Catchable fatal error: Argument 1 passed to TYPO3\\CMS\\Core\\Log\\Writer\\FileWriter::__construct() must be of the type array, null given, called in /home/XXX/typo3_src-6.2.31/typo3/sysext/core/Classes/Utility/GeneralUtility.php on line 4471 and defined in /home/XXX/typo3_src-6.2.31/typo3/sysext/core/Classes/Log/Writer/FileWriter.php on line 62

After removing the snippet you send me yesterday the site is online again. But now the log file is growing every minute again :-(

[hannesbochmann]

Sorry, the snippet I send you was for TYPO3 7.6 upwards. I checked again in one of my 6.2 projects. That's what I put into AdditionalConfiguration.php:

$GLOBALS['TYPO3_CONF_VARS']['LOG']['writerConfiguration'][\TYPO3\CMS\Core\Log\LogLevel::ERROR]['TYPO3\\CMS\\Core\\Log\\Writer\\FileWriter'] =
    $GLOBALS['TYPO3_CONF_VARS']['LOG']['writerConfiguration'][\TYPO3\CMS\Core\Log\LogLevel::DEBUG]['TYPO3\\CMS\\Core\\Log\\Writer\\FileWriter'];
unset($GLOBALS['TYPO3_CONF_VARS']['LOG']['writerConfiguration'][\TYPO3\CMS\Core\Log\LogLevel::DEBUG]['TYPO3\\CMS\\Core\\Log\\Writer\\FileWriter']);

This works since weeks.

[Batman777]

Thank you very much again :-)

[leslawp]

Thank you guys for help, especially Hannes! I went into trouble too with huge, few GB log.

[leslawp]

Actually it resolves (in T3 7.6) problem with lot of DEBUG entries, caused by every view, but if there is somewhere in the world strange link to our page with bad L parameter then Google will index the whole site since this bad L parameter will glue to every single link. Then lot of ERROR labeled entries will go to the log file. And voila, you have again few GB in few minutes: This is apache log that shows how it goes: 66.249.64.207 - - [10/Nov/2017:16:07:03 +0100] "GET /index.php?id=1128&L=11 HTTP/1.1" 200 8479 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" What to do? We need different approach... some kind of redirect on bad L or something like this.

[mediaessenz]

As an other workaround for TYPO3 7.6 and above, you can add this to your AdditionalConfiguration.php:

if (\TYPO3\CMS\Core\Utility\GeneralUtility::getApplicationContext()->isProduction()) {
    $GLOBALS['TYPO3_CONF_VARS'] = array_replace_recursive($GLOBALS['TYPO3_CONF_VARS'],
        [
            'LOG' => [
                'writerConfiguration' => [
                    \TYPO3\CMS\Core\Log\LogLevel::DEBUG => \TYPO3\CMS\Core\Log\Writer\NullWriter::class,
                    \TYPO3\CMS\Core\Log\LogLevel::INFO => \TYPO3\CMS\Core\Log\Writer\NullWriter::class,
                    \TYPO3\CMS\Core\Log\LogLevel::NOTICE => \TYPO3\CMS\Core\Log\Writer\NullWriter::class,
                    \TYPO3\CMS\Core\Log\LogLevel::WARNING => \TYPO3\CMS\Core\Log\Writer\NullWriter::class
                ]
            ]
        ]
    );
}

or change the writer configuration just for the Realurl-Url-Encoder class:

$GLOBALS['TYPO3_CONF_VARS']['LOG']['DmitryDulepov']['Realurl']['Encoder']['UrlEncoder']['writerConfiguration'] = [
    \TYPO3\CMS\Core\Log\LogLevel::ERROR => [
        \TYPO3\CMS\Core\Log\Writer\NullWriter::class => [],
    ],
];

Additionally, if your side doesn't use multiple languages, you could also remove the L parameter from your TypoScript setup:

config.linkVars =

If you use other linkVars, of course you should leave them as before.

[leslawp]

Thank you, @mediaessenz ! I will try that!

[leslawp]

Looks like it works :) Still we need to think about steady solution, since we are doing workarounds...

[neotrace]

I am also experiencing this problem in TYPO3 7.6. Server eventually ran out of space and crashed due to a 60GB Log file full of these RealURL Errors. I have L(0-6) set in my config since I only have those languages, but I see requests for L=10/11/12 and so on coming from outside. Probably Searchengine or bot trying languages. Our Webhoster made a symlink to /dev/null for the log file so that is another working solution I wanted to share here. Waiting desperately for a fix.

[webian]

The huge logs are generated from here: https://github.com/dmitryd/typo3-realurl/blob/development/Classes/Encoder/UrlEncoder.php#L1589 That debug_backtrace() returns an array with more or less 30 elements and each of them have inside the whole \DmitryDulepov\Realurl\Encoder\UrlEncoder object and are later json encoded. This is repeated for every link in the page generating a ~250MB file for each page.

[webian]

I made this rewrite rule for .htaccess to remove unwanted L parameters: RewriteCond %{QUERY_STRING} ^(.)&?L=(?!(1|11)(&|$))[^&]+&?(.)$ RewriteRule ^/?(.*)$ /$1?%1%4 [R=301,L]

Play with the regex: https://regex101.com/r/KsOYvR/1

In this case I allow only L=1 and L=11. Change 1|11 with the ID that you want to allow, i.e. 2|4|5|15, all other values will cause the L parameter to be removed.

[osahner]

551 fixes the Logfile overflow.

I wonder how long we have to wait for a release ...

[fazzyx]

@webian And what if u have partial translated content?

[webian]

@fazzyx sorry I can't understand what you're asking. Can you explain your doubts further?

[fazzyx]

@webian Basically you translate to another language, but not every content / site.

[webian]

@fazzyx my rewrite rule should be used to remove L parameter with values not configured. Does your case have urls with values for L parameter not configured? ie.: you use default language English (uid=0) and a second language german (uid=1). Then you change the regex to allow only L=0 or L=1: ^(.)&?L=(?!(0|1)(&|$))[^&]+&?(.)$. Did I miss something?

[fazzyx]

@webian I just described a scenario where not every content or page are translated. You have only a workaround more, but we need a solution.

[franzkugelmann]

we were able to stop the logging with TypoScript config.linkVars := addToList(L(0|1|4)) You need to add your language-ids instead of 0|1|4. Then TYPO3 does not let the not-existing language id through and the logger in realurl is not triggered.

[LeoniePhiline]

+1. We are also badly affected by this bug, with debug_backtrace() filling up Gigabytes of logs until the server hard drive runs out and the site shuts down. This should seriously be fixed (although indeed config.linkVars must be fixed by everyone coming across this bug). In any case, debug_backtrace() is not even helping, but making things worse. Instead of using debug_backtrace(), config.linkVars should be logged with a hint to verify the valid L-values are up to date.

[fazzyx]

@franzkugelmann As I understand this, It is only working, if every page is translated but not if you have a partially translated environment. If you have setup translation to a foreign language, but not every page is translated to this language you need the linkVars id for this language anyway and therefor it is possible to call a page which is not translated with one of the defined linkVars id's which in this case is invalid for this page.

[franzkugelmann]

@fazzyx Yes, you are right. config.linkVars only helps with non-existing languages (as defined in table sys_language), but not with just-not-translated pages. Our problems occured due to a deleted language which was not supported by the editors anymore.

[Bjoelin]

I am currently playing around with the configuration and just realized; this als ooccurs with "correct" L parameters.

I my case, 3 is set as sys_language_uid for Dutch content. Setting config.defaultGetVars.L = 3 breaks url rewriting and produces huge logs as well.

Bad "L" parameter ("3") was detected by realurl.

Can anyone confirm this problem?

[dmitryd]

Please, do not use debug logging in production. That's all.

[dmitryd]

@Bjoelin This warning is produced if you have no language record with uid=3 in your system.

[franzkugelmann]

A hint about the production context dmitry mentioned: If you wonder why you get the big dev context logging on your production site, check your devIpMask! See the description in the install tool at [SYS][debugExceptionHandler] We had the customers IP in there and thought this perfectly valid. Until they started running a SEO crawler, which resulted in thousands of requests triggering the debugExceptionHandler.

[auge]

for TYPO3-8.7 I had to pass an array instead of the classname:

$GLOBALS['TYPO3_CONF_VARS']['LOG']['writerConfiguration'][\TYPO3\CMS\Core\Log\LogLevel::WARNING]['TYPO3\\CMS\\Core\\Log\\Writer\\FileWriter'] = array(\TYPO3\CMS\Core\Log\Writer\NullWriter::class => array())

the key of the array is the classname to be factored, the value are options (just empty array here).

[pniederlag]

This is really an issue as an invalid L parameter is not that unlikely to happen.

Removing/Unsetting the Log or setting it to the NullWriter does work of course but actually is more of a workarund.

@dmitryd could you please remove the debug_backtrace() from the error message? It really quickly fills the disk with hundreds of GB logs! (we have ~250MB per single request with a broken L parameter).

My suggestions/pledge is:

• lower priority to warning
• add some useful information but not a full backtrace

If you are willing to accept a PR I'd be happy to prepare one.

nonetheless, thx for your great work, Peter

[pniederlag]

@dmitryd

Please, do not use debug logging in production. That's all.

The problem does occur/happen with production exceptionhandlers and all other relevant settings set for a production context.

$this->logger->error($message, debug_backtrace());

In general you'd want to catch errors in production settings, however you'd not want it to contain result of a call to debug_backtrace() ;)

[LeoniePhiline]

@pniederlag +1

[nabossha]

+1 at least in production-context it should be reduced to a simple error-message

[argerweb]

@pniederlag +1

[mhelwig]

This effectively means that any simple security scanner can shoot down your Typo3 page in a normal server environment within minutes. It's a severe DoS vulnerability. One of my servers generated approx. 450 MB Iogs for each invalid request. I also reported this to the Typo3 security team a while ago but since it's already public they don't seem to care.

[dmitryd]

Should be fixed now.

[outdoorsman]

I'll check out that fix... thanks for that @dmitryd. This took down a trove of our websites in a shared environment because of a single massive log file.

[Batman777]

So has RealURL 2.3.1 fixed that issue ?

[outdoorsman]

@Batman777 I've installed 2.3.1 and am getting a much abbreviated log error compared to what I was getting. But am still getting these two for every page load...

Sun, 07 Jan 2018 16:31:30 -0800 [ERROR] request="b098b6f4ff873" component="DmitryDulepov.Realurl.Encoder.UrlEncoder": Bad "L" parameter ("1") was detected by realurl. Page caching is disabled to prevent spreading of wrong "L" value.
Sun, 07 Jan 2018 16:31:30 -0800 [ERROR] request="b098b6f4ff873" component="DmitryDulepov.Realurl.Encoder.UrlEncoder": Bad "L" parameter ("2") was detected by realurl. Page caching is disabled to prevent spreading of wrong "L" value.

I'm surprised that I'm having an issue with this at all because I only have one language set. The only place I've found reference other languages is conditionals within the bootstrap_package, could this be triggering the additional L parameters?

[globalVar = GP:L = 1]
    config {
        sys_language_uid = 1
        language = de
        locale_all = de_DE.UTF-8
        htmlTag_setParams = lang="de" dir="ltr" class="no-js"
    }
[global]
[globalVar = GP:L = 2]
    config {
        sys_language_uid = 2
        language = da
        locale_all = da_DK.UTF-8
        htmlTag_setParams = lang="da" dir="ltr" class="no-js"
    }
[global]

[dmitryd]

@Outdoorsman

Check if your config has config.linkVars = L(0-2) (it should!). Also check for addQueryString in TypoScript and viewHelpers.

Also read this: https://github.com/dmitryd/typo3-realurl/wiki/Notes-for-Integrators#i-see-a-lot-of-strange-entries-in-the-realurl-module

[adrienjacob]

There's a typical issue with bootstrap_package : you need to disable unneeded languages in the template constants - page.theme.language.languageValue = 0 (if you only use the default language)

cf. this other thread : https://github.com/dmitryd/typo3-realurl/issues/407#issuecomment-285704553

[wtfred]

@Outdoorsman Did you get the 2.3.1 from TYPO3 TER or directly from github ? I made the update from the TER but i'm still getting huge log files...

[sgrossberndt]

2.3.1 does not contain the fix: https://github.com/dmitryd/typo3-realurl/commit/2c669925ebdec3212d01996b79507bdaba29db11#commitcomment-26436691

[outdoorsman]

When I first updated it, it seemed to keep spewing log files that went in the the gigabytes within minutes. But if I remember right (and I'm not sure about this), I cleared cache from the TYPO3 backend and it didn't work but then cleared them from the Install Tool which I believe does it much more thoroughly and then it reduced down to single line entries instead of a full trace... at least manageable if you don't leave it too long. I used the Development branch from... https://github.com/dmitryd/typo3-realurl.git. You should be in Production/Live mode too.

[franzholz]

I do not understand the wiki page https://github.com/dmitryd/typo3-realurl/wiki/Notes-for-Integrators#i-see-a-lot-of-strange-entries-in-the-realurl-module Why are these strange L parameters generated from anywhere inside of TYPO3? I thought all links are generated by realurl if activated. Therefore these links with wrong L parameters should not exist at all.

[08-Jan-2018 19:27:02 Europe/Berlin] http://mywebsite.com/ - cms: $TSFE->set_no_cache() was triggered. Reason: Bad "L" parameter ("courier-authlib-0.59.1-1.x86_64.rpm") was detected by realurl. Page caching is disabled to prevent spreading of wrong "L" value.. Caching is disabled!

If I use the TYPO3 backend to search for a term "courier-authlib-0.59.1-1.x86_64", then I have no result. I have made a search on the filesystem. No file "courier-authlib-0.59.1-1.x86_64.rpm" exists. It has been removed many years ago.