Many Errors - WordPress database error Duplicate entry

Mikeekb commented 4 years ago

[x] I've read and understood the contribution guidelines.
[x] I've searched for any related issues and avoided creating a duplicate issue.

Please give us a description of what happened.

There are so many errors that php-errors.log file has a size of 10GB in 2 hours. Log in 2 seconds https://yadi.sk/d/NMqZRAT02IzwpA Perhaps this is due to our multi-domain site with the same Wordpress core. This was discussed in detail here https://wordpress.org/support/topic/php-errors-40-gigabytes-in-size/ Please take a look. And this is due to the new indexing function to speed up the site

Please describe what you expected to happen and why.

I expect to see a hook in the next version with which I can disable indexing. So that while the problem is being solved, my site works correctly. Now, due to these errors, there is an excess of CPU and RAM, so the site slows down a lot. Screenshot https://yadi.sk/i/hFZjt5iA7PcKmQ

How can we reproduce this behavior?

If the problem is in multidomains, then like this:

Add the following lines to wp_config.php:

$root_domain = 'your_domain.com';
$this_domain = $_SERVER['SERVER_NAME'];
if (!preg_match("#^([a-zA-Z0-9-]+\.)?{$root_domain}$#",$this_domain)) {
echo "ERROR: The domain [$this_domain] is not a valid domain for this website.";
die();
} else {
define('WP_HOME',"https://{$this_domain}");
define('WP_SITEURL',"https://{$this_domain}");
}

Add subdomains to the main domain (DNS A records), for example, test1.your_domain.com and test2.your_domain.com
Indexing of pages from different domains should start, I don’t know how indexing works, perhaps there should be links to pages from other subdomains.

Technical info

If relevant, which editor is affected (or editors):
[x] Classic Editor
[ ] Gutenberg
[ ] Classic Editor plugin

Which browser is affected (or browsers):
[x] Chrome
[x] Firefox
[x] Safari
[ ] Other

Used versions

WordPress version: 5.5.1
Yoast SEO version: 15.0
Gutenberg plugin version: no
Classic Editor plugin version: no
Relevant plugins in case of a bug: Disabled all plugins except Woocommerce and Yoast SEO - problem remains
Tested with theme: my theme and storefront (the problem exists on both theme);

Djennez commented 4 years ago

@Mikeekb thanks for the report. I haven't reproduced it yet, hopefully will get to that sometime soon.

In the meantime, maybe you can check these pointers:

Every entry in the log seems to mention your custom theme ndt-group. I wonder if the logs look different is you switch to a default theme like twentytwenty. Could you check that?
indexables is core functionality of our plugin, there will be no filters to disable this as this is needed for our plugin to work.
Looking at the errors, I think your main problem is your disk space? In that case you might want to dial back your logging verbosity. It's good to have such verbose logs for debugging, but it's not really practical in a live environment and you can do with just partial stacktraces there. Saves on discspace and IO. I don't think the errors themselves will impede the workings of your website / our plugins' functionality too much at the moment.

Mikeekb commented 4 years ago

@Djennez Hi

Every entry in the log seems to mention your custom theme ndt-group. I wonder if the logs look different is you switch to a default theme like twentytwenty. Could you check that?

Yes, I tried another theme - Storefront, since I have Woocommerce. And disabled all plugins except Yoast SEO and Woocommerce. Here's how to get a log https://yadi.sk/d/8RWNYCMMmN4WJA there is also a mention of the theme, but now storefront require_once('/themes/storefront/header.php') This didn't fix the problem.

indexables is core functionality of our plugin, there will be no filters to disable this as this is needed for our plugin to work.

I understood you, but yesterday I rolled back the Yoast version to 14.8.1, the hosting load became much lower, see the screenshot https://yadi.sk/i/udC44MMNhYyBew and this version has an indexing function. A similar error occurs, but much less often, here is a new log https://yadi.sk/d/_wwLzxJhTqV2zQ only 2 errors per day.

Looking at the errors, I think your main problem is your disk space?

Yes, I can reduce the detail of the log, but due to errors, the site slows down a lot, and sometimes error 503 appears, due to the fact that the server is not coping.

Djennez commented 4 years ago

Is this issue still around with 15.1.1?

Mikeekb commented 4 years ago

@Djennez Yes, the problem persists. The log contains exactly the same errors. And with version 15.1.1 my other site completely froze. PHP Timeout errors are coming out. Log from another site with which this problem is https://yadi.sk/i/5PSM0hqA_ymMPg If I turn off Yoast SEO, then everything is fine.

Djennez commented 4 years ago

Yea those errors are not from our plugin but from the clearfy plugin...

Please try to deactivate all plugins, update Yoast to 15.1.1 and see if it works without issues. If it does, start reactivating plugins and see when the problems start.

Mikeekb commented 4 years ago

On a site that completely freezes with version 15.1.1, I've tried disabling all plugins except Yoast SEO and enabling the Twenty Twenty theme, the problem remains.

Yea those errors are not from our plugin but from the clearfy plugin...

No, look at other lines, there and the kernel scripts hang. It's just that any script starts to freeze after enabling Yoast. For example /wp-includes/rewrite.php on line 571

Djennez commented 4 years ago

On a site that completely freezes with version 15.1.1, I've tried disabling all plugins except Yoast SEO and enabling the Twenty Twenty theme, the problem remains.

Unfortunately, this seems to be a rare / case-specific issue, so there is something going on on your site / server that would cause this to happen. I'm afraid we can't investigate what that something is, as we don't have your setup.

No, look at other lines, there and the kernel scripts hang. It's just that any script starts to freeze after enabling Yoast. For example /wp-includes/rewrite.php on line 571

You may be able to gather data from your host as to what the code execution is doing in this time.

If you're up for it, you might also try this on a staging environment! If you know what you're doing:

Create a backup of your database, perform this on a staging site that exhibits the same problems!
Deactivate all plugins, including Yoast.
Remove the tables: _yoast_seo_links, _yoast_indexable_hierarchy, _yoast_indexable, wp_yoast_migrations and _yoast_prominent_words (if it exists).
In the _options table, search for the option_name: yoast_migrations_free and / or yoast_migrations_premium and remove these lines.
Install and activate Yoast 15.1.1
How is it working now?

Mikeekb commented 4 years ago

I made a copy of the site to painlessly disable plugins and change the theme. And also removed tables and option as you wrote https://yadi.sk/i/Yt-FhC-KxvJ81Q Watch the video, the problem persists https://yadi.sk/d/rmfvH2Lwbwlgkg

Mikeekb commented 4 years ago

Latest log https://yadi.sk/d/3rDrEojoH1zSQA

Djennez commented 4 years ago

How much memory does an average request use without Yoast enabled?

Mikeekb commented 4 years ago

Set memory_limit = 8M, site works correctly without Yoast SEO plugin https://yadi.sk/i/FwIQpw8gVsR10w

Djennez commented 4 years ago

Not sure if that amounts to the same sort of check, but sounds reasonable. But then I don't have any other options for you to check unfortunately. It may be database(content) related. Based on the very first log you shared, there seems to be a continuous loop happening but I can't say for sure why.

Mikeekb commented 4 years ago

I can send you an e-mail access to the site ( FTP, Admin panel and PHPMyAdmin ), can you see what the problem is?

Yoast / wordpress-seo