search

highlightjs / highlight.js

JavaScript syntax highlighter with language auto-detection and zero dependencies.
https://highlightjs.org/
BSD 3-Clause "New" or "Revised" License
23.31k stars 3.52k forks source link

Syntax error: Unexpected \ in line 378 #3988

Closed jurgenhaas closed 4 months ago

jurgenhaas commented 4 months ago

We are using the highlight.min.js version v11.9.0 as part of a Drupal theme. Drupal core has a fairly good asset optimizer and send all css and js files through a process that also parses the input.

With this highlight file, that process fails with the message Syntax error: Unexpected \, File: themes/custom/bst/js/highlight.min.js, Line: 378, Column: 39, Index: 34322. The stack trace looks like this:

#0 /var/www/html/vendor/mck89/peast/lib/Peast/Syntax/Scanner.php(911): Peast\Syntax\Scanner->error()
#1 /var/www/html/vendor/mck89/peast/lib/Peast/Syntax/Scanner.php(800): Peast\Syntax\Scanner->getToken()
#2 /var/www/html/vendor/mck89/peast/lib/Peast/Syntax/Scanner.php(546): Peast\Syntax\Scanner->getNextToken()
#3 /var/www/html/vendor/mck89/peast/lib/Peast/Syntax/Parser.php(2778): Peast\Syntax\Scanner->getState()
#4 /var/www/html/vendor/mck89/peast/lib/Peast/Syntax/Parser.php(3055): Peast\Syntax\Parser->parseArrowFunction()
#5 /var/www/html/vendor/mck89/peast/lib/Peast/Syntax/ParserAbstract.php(213): Peast\Syntax\Parser->parseAssignmentExpression()
#6 /var/www/html/vendor/mck89/peast/lib/Peast/Syntax/Parser.php(3548): Peast\Syntax\ParserAbstract->isolateContext()
#7 /var/www/html/vendor/mck89/peast/lib/Peast/Syntax/Parser.php(3696): Peast\Syntax\Parser->parseArrayLiteral()
#8 /var/www/html/vendor/mck89/peast/lib/Peast/Syntax/Parser.php(3351): Peast\Syntax\Parser->parsePrimaryExpression()
#9 /var/www/html/vendor/mck89/peast/lib/Peast/Syntax/Parser.php(3279): Peast\Syntax\Parser->parseLeftHandSideExpression()
#10 /var/www/html/vendor/mck89/peast/lib/Peast/Syntax/Parser.php(3230): Peast\Syntax\Parser->parsePostfixExpression()
#11 /var/www/html/vendor/mck89/peast/lib/Peast/Syntax/Parser.php(3141): Peast\Syntax\Parser->parseUnaryExpression()
#12 /var/www/html/vendor/mck89/peast/lib/Peast/Syntax/Parser.php(3103): Peast\Syntax\Parser->parseLogicalBinaryExpression()
#13 /var/www/html/vendor/mck89/peast/lib/Peast/Syntax/Parser.php(3059): Peast\Syntax\Parser->parseConditionalExpression()
#14 /var/www/html/vendor/mck89/peast/lib/Peast/Syntax/ParserAbstract.php(213): Peast\Syntax\Parser->parseAssignmentExpression()
#15 /var/www/html/vendor/mck89/peast/lib/Peast/Syntax/Parser.php(2864): Peast\Syntax\ParserAbstract->isolateContext()
#16 /var/www/html/vendor/mck89/peast/lib/Peast/Syntax/Parser.php(2826): Peast\Syntax\Parser->parsePropertyDefinition()
#17 /var/www/html/vendor/mck89/peast/lib/Peast/Syntax/Parser.php(3698): Peast\Syntax\Parser->parseObjectLiteral()
#18 /var/www/html/vendor/mck89/peast/lib/Peast/Syntax/Parser.php(3351): Peast\Syntax\Parser->parsePrimaryExpression()
#19 /var/www/html/vendor/mck89/peast/lib/Peast/Syntax/Parser.php(3279): Peast\Syntax\Parser->parseLeftHandSideExpression()
#20 /var/www/html/vendor/mck89/peast/lib/Peast/Syntax/Parser.php(3230): Peast\Syntax\Parser->parsePostfixExpression()
#21 /var/www/html/vendor/mck89/peast/lib/Peast/Syntax/Parser.php(3141): Peast\Syntax\Parser->parseUnaryExpression()
#22 /var/www/html/vendor/mck89/peast/lib/Peast/Syntax/Parser.php(3103): Peast\Syntax\Parser->parseLogicalBinaryExpression()
#23 /var/www/html/vendor/mck89/peast/lib/Peast/Syntax/Parser.php(3059): Peast\Syntax\Parser->parseConditionalExpression()
#24 /var/www/html/vendor/mck89/peast/lib/Peast/Syntax/Parser.php(2921): Peast\Syntax\Parser->parseAssignmentExpression()
#25 /var/www/html/vendor/mck89/peast/lib/Peast/Syntax/Parser.php(1912): Peast\Syntax\Parser->parseInitializer()
#26 /var/www/html/vendor/mck89/peast/lib/Peast/Syntax/ParserAbstract.php(349): Peast\Syntax\Parser->parseVariableDeclaration()
#27 /var/www/html/vendor/mck89/peast/lib/Peast/Syntax/Parser.php(1843): Peast\Syntax\ParserAbstract->charSeparatedListOf()
#28 /var/www/html/vendor/mck89/peast/lib/Peast/Syntax/ParserAbstract.php(213): Peast\Syntax\Parser->parseLexicalDeclaration()
#29 /var/www/html/vendor/mck89/peast/lib/Peast/Syntax/Parser.php(438): Peast\Syntax\ParserAbstract->isolateContext()
#30 /var/www/html/vendor/mck89/peast/lib/Peast/Syntax/Parser.php(357): Peast\Syntax\Parser->parseDeclaration()
#31 /var/www/html/vendor/mck89/peast/lib/Peast/Syntax/Parser.php(338): Peast\Syntax\Parser->parseStatementListItem()
#32 [internal function]: Peast\Syntax\Parser->parseStatementList()
#33 /var/www/html/vendor/mck89/peast/lib/Peast/Syntax/ParserAbstract.php(213): call_user_func_array()
#34 /var/www/html/vendor/mck89/peast/lib/Peast/Syntax/Parser.php(1634): Peast\Syntax\ParserAbstract->isolateContext()
#35 /var/www/html/vendor/mck89/peast/lib/Peast/Syntax/ParserAbstract.php(213): Peast\Syntax\Parser->parseFunctionBody()
#36 /var/www/html/vendor/mck89/peast/lib/Peast/Syntax/Parser.php(1465): Peast\Syntax\ParserAbstract->isolateContext()
#37 /var/www/html/vendor/mck89/peast/lib/Peast/Syntax/Parser.php(431): Peast\Syntax\Parser->parseFunctionOrGeneratorDeclaration()
#38 /var/www/html/vendor/mck89/peast/lib/Peast/Syntax/Parser.php(357): Peast\Syntax\Parser->parseDeclaration()
#39 /var/www/html/vendor/mck89/peast/lib/Peast/Syntax/Parser.php(338): Peast\Syntax\Parser->parseStatementListItem()
#40 [internal function]: Peast\Syntax\Parser->parseStatementList()
#41 /var/www/html/vendor/mck89/peast/lib/Peast/Syntax/ParserAbstract.php(213): call_user_func_array()
#42 /var/www/html/vendor/mck89/peast/lib/Peast/Syntax/Parser.php(1634): Peast\Syntax\ParserAbstract->isolateContext()
#43 /var/www/html/vendor/mck89/peast/lib/Peast/Syntax/ParserAbstract.php(213): Peast\Syntax\Parser->parseFunctionBody()
#44 /var/www/html/vendor/mck89/peast/lib/Peast/Syntax/Parser.php(1538): Peast\Syntax\ParserAbstract->isolateContext()
#45 /var/www/html/vendor/mck89/peast/lib/Peast/Syntax/Parser.php(3688): Peast\Syntax\Parser->parseFunctionOrGeneratorExpression()
#46 /var/www/html/vendor/mck89/peast/lib/Peast/Syntax/Parser.php(3351): Peast\Syntax\Parser->parsePrimaryExpression()
#47 /var/www/html/vendor/mck89/peast/lib/Peast/Syntax/Parser.php(3279): Peast\Syntax\Parser->parseLeftHandSideExpression()
#48 /var/www/html/vendor/mck89/peast/lib/Peast/Syntax/Parser.php(3230): Peast\Syntax\Parser->parsePostfixExpression()
#49 /var/www/html/vendor/mck89/peast/lib/Peast/Syntax/Parser.php(3141): Peast\Syntax\Parser->parseUnaryExpression()
#50 /var/www/html/vendor/mck89/peast/lib/Peast/Syntax/Parser.php(3103): Peast\Syntax\Parser->parseLogicalBinaryExpression()
#51 /var/www/html/vendor/mck89/peast/lib/Peast/Syntax/Parser.php(3059): Peast\Syntax\Parser->parseConditionalExpression()
#52 /var/www/html/vendor/mck89/peast/lib/Peast/Syntax/Parser.php(2921): Peast\Syntax\Parser->parseAssignmentExpression()
#53 /var/www/html/vendor/mck89/peast/lib/Peast/Syntax/Parser.php(1912): Peast\Syntax\Parser->parseInitializer()
#54 /var/www/html/vendor/mck89/peast/lib/Peast/Syntax/ParserAbstract.php(349): Peast\Syntax\Parser->parseVariableDeclaration()
#55 /var/www/html/vendor/mck89/peast/lib/Peast/Syntax/Parser.php(1897): Peast\Syntax\ParserAbstract->charSeparatedListOf()
#56 /var/www/html/vendor/mck89/peast/lib/Peast/Syntax/ParserAbstract.php(213): Peast\Syntax\Parser->parseVariableDeclarationList()
#57 /var/www/html/vendor/mck89/peast/lib/Peast/Syntax/Parser.php(1874): Peast\Syntax\ParserAbstract->isolateContext()
#58 /var/www/html/vendor/mck89/peast/lib/Peast/Syntax/Parser.php(381): Peast\Syntax\Parser->parseVariableStatement()
#59 /var/www/html/vendor/mck89/peast/lib/Peast/Syntax/Parser.php(359): Peast\Syntax\Parser->parseStatement()
#60 /var/www/html/vendor/mck89/peast/lib/Peast/Syntax/Parser.php(338): Peast\Syntax\Parser->parseStatementListItem()
#61 /var/www/html/vendor/mck89/peast/lib/Peast/Syntax/Parser.php(199): Peast\Syntax\Parser->parseStatementList()
#62 /var/www/html/web/core/lib/Drupal/Core/Asset/JsOptimizer.php(49): Peast\Syntax\Parser->parse()
#63 /var/www/html/web/core/lib/Drupal/Core/Asset/JsCollectionOptimizerLazy.php(178): Drupal\Core\Asset\JsOptimizer->optimize()
#64 /var/www/html/web/core/modules/system/src/Controller/AssetControllerBase.php(183): Drupal\Core\Asset\JsCollectionOptimizerLazy->optimizeGroup()
#65 [internal function]: Drupal\system\Controller\AssetControllerBase->deliver()
#66 /var/www/html/web/core/lib/Drupal/Core/EventSubscriber/EarlyRenderingControllerWrapperSubscriber.php(123): call_user_func_array()
#67 /var/www/html/web/core/lib/Drupal/Core/Render/Renderer.php(627): Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->Drupal\Core\EventSubscriber\{closure}()
#68 /var/www/html/web/core/lib/Drupal/Core/EventSubscriber/EarlyRenderingControllerWrapperSubscriber.php(124): Drupal\Core\Render\Renderer->executeInRenderContext()
#69 /var/www/html/web/core/lib/Drupal/Core/EventSubscriber/EarlyRenderingControllerWrapperSubscriber.php(97): Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->wrapControllerExecutionInRenderContext()
#70 /var/www/html/vendor/symfony/http-kernel/HttpKernel.php(181): Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->Drupal\Core\EventSubscriber\{closure}()
#71 /var/www/html/vendor/symfony/http-kernel/HttpKernel.php(76): Symfony\Component\HttpKernel\HttpKernel->handleRaw()
#72 /var/www/html/web/core/lib/Drupal/Core/StackMiddleware/Session.php(58): Symfony\Component\HttpKernel\HttpKernel->handle()
#73 /var/www/html/web/core/lib/Drupal/Core/StackMiddleware/KernelPreHandle.php(48): Drupal\Core\StackMiddleware\Session->handle()
#74 /var/www/html/web/core/lib/Drupal/Core/StackMiddleware/ContentLength.php(28): Drupal\Core\StackMiddleware\KernelPreHandle->handle()
#75 /var/www/html/web/core/modules/big_pipe/src/StackMiddleware/ContentLength.php(32): Drupal\Core\StackMiddleware\ContentLength->handle()
#76 /var/www/html/web/core/modules/page_cache/src/StackMiddleware/PageCache.php(106): Drupal\big_pipe\StackMiddleware\ContentLength->handle()
#77 /var/www/html/web/core/modules/page_cache/src/StackMiddleware/PageCache.php(85): Drupal\page_cache\StackMiddleware\PageCache->pass()
#78 /var/www/html/web/core/modules/ban/src/BanMiddleware.php(50): Drupal\page_cache\StackMiddleware\PageCache->handle()
#79 /var/www/html/web/modules/contrib/crowdsec/src/Middleware.php(80): Drupal\ban\BanMiddleware->handle()
#80 /var/www/html/web/core/lib/Drupal/Core/StackMiddleware/ReverseProxyMiddleware.php(48): Drupal\crowdsec\Middleware->handle()
#81 /var/www/html/web/core/lib/Drupal/Core/StackMiddleware/NegotiationMiddleware.php(51): Drupal\Core\StackMiddleware\ReverseProxyMiddleware->handle()
#82 /var/www/html/web/core/lib/Drupal/Core/StackMiddleware/AjaxPageState.php(36): Drupal\Core\StackMiddleware\NegotiationMiddleware->handle()
#83 /var/www/html/web/core/lib/Drupal/Core/StackMiddleware/StackedHttpKernel.php(51): Drupal\Core\StackMiddleware\AjaxPageState->handle()
#84 /var/www/html/web/core/lib/Drupal/Core/DrupalKernel.php(704): Drupal\Core\StackMiddleware\StackedHttpKernel->handle()
#85 /var/www/html/web/index.php(19): Drupal\Core\DrupalKernel->handle()
#86 {main}

The offending line appears to be this one:

match:[/const|var|let/,/\s+/,t,/\s*/,/=\s*/,/(async\s*)?/,n.lookahead(x)],
joshgoebel commented 4 months ago

We really need to be shown something is actually wrong with our library (as we ship it) to assist. Right now it just looks like Drupal's optimizer is broken (however reliable it typically is) - and it's unclear why.

The only\ present in that line all appear to be valid regex escapes.

jurgenhaas commented 4 months ago

It's probably more the mck89/peast/lib/Peast/Syntax/Scanner.php which is used by Drupal. I'd guess, that the research would be between the two components to see what's wrong.

For our current theme, we've "fixed" this by not optimizing highlight.js - so that works. Whether the issue between highlight.js and mck89/peast should be addressed or not remains to be decided.

joshgoebel commented 4 months ago

Unless you can show that our JS is somehow actually invalid it's definitely not an issue with Highlight.js. The fact that its parsed by all modern browsers without difficultly would seem to suggest its valid. I'd suggest filing a support issue against the scanner project if you want to get to the bottom of this.