search

eliashaeussler / typo3-warming

🚀 Extension for TYPO3 CMS that warms up Frontend caches based on an XML sitemap with multi-language support.
https://docs.typo3.org/p/eliashaeussler/typo3-warming/main/en-us/
GNU General Public License v2.0
15 stars 9 forks source link

[FEATURE] Support Basic Auth for frontend requests #85

Closed josefglatz closed 2 years ago

josefglatz commented 2 years ago

Is your feature request related to a problem?

IN non productive environments it would be nice to have the possibility to setup basic auth credentials

Describe the solution you'd like

Setting user and password for frontend requests would be nice. Also setting it via an option like --foo=user:pw would be nice.

Describe alternatives you've considered

No response

Additional context

No response

Code of Conduct

Tim-Obert commented 2 years ago

A possible workaround would be to configure your vhost so that the basic auth is only needed if the request is not from your server. For example in Apache:

    AuthType Basic
    AuthName "Please enter your username and password"
    AuthUserFile /var/www/files/.htpasswd

    <RequireAny>
      Require ip <public ip of your server> 
      Require valid-user
    </RequireAny>
eliashaeussler commented 2 years ago

Hi @josefglatz, thanks for your issue.

With eliashaeussler/cache-warmup#95 (released as 0.8.2) and #118 of EXT:warming (released as 0.4.4) you can now provide custom client configuration. It will be used in the default crawlers to instantiate the Guzzle client.

You can use the following JSON as extension configuration crawlerOptions and verboseCrawlerOptions:

{
    "client_config": {
        "auth": {
            "<username>",
            "<password>"
        }
    }
}

⚠️ Make sure the latest supported versions of both EXT:warming and the mentioned upstream package are installed. You can find more information in them mentioned PR #118 as well.

josefglatz commented 2 years ago

How should I pass this via AdditionalConfiguration.php? Do I have to use json_encode()?

eliashaeussler commented 2 years ago

How should I pass this via AdditionalConfiguration.php? Do I have to use json_encode()?

Yes, you need to configure a JSON-encoded string for the crawlerOptions and verboseCrawlerOptions extension configurations. If this is configured via AdditionalConfiguration.php, then you probably should do something like that:

$GLOBALS['TYPO3_CONF_VARS']['EXTENSIONS']['warming']['crawlerOptions'] = json_encode([
    'client_config' => [
        'auth' => [
            '<username>',
            '<password>',
        ],
    ],
]);
josefglatz commented 9 months ago

I'm not sure if this options works correct.

TYPO3 11.5.latest stable with following resulting config (resulting config via ./vendor/bin/typo3 configuration:show EXTENSIONS)):

        'warming' => [
            'crawler' => 'EliasHaeussler\\Typo3Warming\\Crawler\\ConcurrentUserAgentCrawler',
            'crawlerOptions' => '{"client_config":{"auth":["foo","bar"]}}',
            'enablePageTree' => '1',
            'enableToolbar' => '1',
            'limit' => '250',
            'supportedDoktypes' => '1',
            'verboseCrawler' => 'EliasHaeussler\\Typo3Warming\\Crawler\\OutputtingUserAgentCrawler',
            'verboseCrawlerOptions' => '',
        ],

Running ./vendor/bin/typo3 warming:cachewarmup -s site-name

results in

Running cache warmup by Elias Häußler and contributors.
Parsing sitemaps...

  [ GuzzleHttp\Exception\ClientException ]
  Client error: `GET https://site-name.lorem.ipsum.at/sitemap.xml` resulted in a `401 Unauthorized` response:
  <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
  <html><head>
  <title>401 Unauthorized</title>
  </head><body>
  <h1>Unauth (truncated...)
eliashaeussler commented 8 months ago

@josefglatz That's most probably an issue with the XmlParser not being configurable in terms of Guzzle client options. The parser requests sitemap.xml and runs into a 401 because of missing basic auth.

I fixed this for newer versions of the extension (see #502) but didn't backport this fix because the implementation for the 0.8.x versions is a little bit tricky. If you're interested to see this feature in the 0.8.x versions, I'd be happy about a sponsoring. And it would be nice if you open a new issue for this topic :)