Crawling sites with basic auth fails authorization

[vladstanca]

When a site is behind basic auth, the resulting static site just has the failed authorization error for every page.

To Reproduce Steps to reproduce the behavior:

1. Add basic auth to site
2. Enter correct credentials in plugin settings
3. Generate static site
4. View static site or html and see auth error

Environment (please complete the following information):

• Hosting OS: Ubuntu 20.04
• Web server setup: php7.4, nginx, provisioned by Laravel Forge
• Hosting company: DigitalOcean

Additional context From what I could find, the issue is with the headers. I fond the following:

The plugin uses code similar to this in Crawler & SitemapParser & DetectSitempasURLs

    $headers = [];
    $headers['auth'] = [ $auth_user, $auth_password ];
    $request = new Request( 'GET', $url, $headers );
    $response = $this->client->send( $request );

new Request() expects actual headers, so Authorization: Basic ###, but the code above is formatted for client options.

Using $response = $this->client->request( 'GET', $url, $headers ); works because the expected 3rd argument is $options and the format of the $headers is ['auth' => ['user', 'pass']].

In my local copy I was able to get it working by making those changes and also changing DetectSitemapsURLs::detect() to pass in the guzzle options to SitemapParser.

        $sitemaps_urls = [];
        $parser = new SitemapParser('WP2Static.com', [
            'strict' => false,
            'guzzle' => ['auth' => [$auth_user, $auth_password]]
        ]);

These changes seemed to work fine on one site, but on another site the plugin kept failing saying wp-sitemap.xml was not found. So i didn't feel comfortable submitting a pull request, but I hope this helps.

[john-shaffer]

This is fixed in develop now. Thanks, @vladstanca and @thecodeassassin!

[web-programmer-here]

@john-shaffer when is this going to be released? all my exported pages, i get Unauthorized message.

[leonstafford]

answered in https://github.com/WP2Static/wp2static/discussions/873