Update to 5.3 and re-index

[bobbingwide]

WordPress 5.3 "Kirk" was released on 12th November 2019. We need to update wp-a2z.org with the new version and re-parse it. Trouble is, there are a couple of outstanding problems:

1. The parse has got confused and can't get/set its status
2. So we have to start it by passing a Git commit point e.g.

   oikwp oik-shortcodes.php wordpress 7d3e363 url=core.wp-a2z.org

3. If it fails then we have to start again, which restarts at the beginning.
4. It's failing when attempting to parse core.

   
   Processing: prepare_item_for_database,WP_REST_Posts_Controller::prepare_item_for_database,
   wp-includes/rest-api/endpoints/class-wp-rest-posts-controller.php

mmap() failed: [12] Cannot allocate memory

Fatal error: Out of memory (allocated 1046478848) (tried to allocate 1052672 bytes) in /home/bworguk/public_html/wp-content/plugins/oik-shortcodes/classes/oik-listapis2.php on line 204

[bobbingwide]

Additionally, the sitemap for https://core.wp-a2z.org, which only contains WordPress core - no plugins or themes - now has twice as many APIs and many more files and classes. Do all of these really need to be parsed for every subdomain? Extract from https://core.wp-a2z.org/sitemap/ etc.

Content type	core.	core.	wp-a2z.org	wp-a2z.org	genesis.	jetpack.
Figures for	5.x?	5.3	5.2.4	5.3	theme	Jetpack
Plugins	1	1	3	3	1	2
Themes	0	0	5	5	1	0
Blocks	65	76	0	0	0	20
Shortcodes	7	7	7	7	26	23
APIs	5838	11776	10061	10927	10584	16645
Files	509	2356	1765	2164	1842	2606
Classes	261	3963	2433	3147	2597	3521
Hooks	2017	3295	4042	4114	3734	6005

Note: wp-a2z.org didn't have a problem with the parse status object. I ran the parse. The above table shows figures for 5.2.4 and 5.3.

[bobbingwide]

Don't forget there's a start parameter. e.g. start from file 567.

oikwp oik-shortcodes.php wordpress 7d3e363 567 url=core.wp-a2z.org

...but it doesn't work in oik-shortcodes v1.35.0.

[bobbingwide]

Hmm, in core.wp-a2z.org there are 12 instances of _excerpt_render_inner_columns_blocks Q. why's that? A. Don't know but there are similar problems in core.wp.a2z with where there's two of just about everything. And the version with the suffixed permalink doesn't exist.

Sourcefile not defined for 45296bw_api cannot determine the funcname

In fact, none of the APIs appear to exist! So that's not a good example.

[bobbingwide]

A. Still don't know but there could be a problem associated with a directory called wp-includes\blocks and a file called blocks.php. Something went wrong for all subdomains in December 2018! I'll completely rebuild my local instance of core.wp.a2z and see what happens.

Post scriptum . No. This is just a red herring.

[bobbingwide]

Oh my. I rebuilt core.wp-a2z.org last night and something went terribly wrong.

Content type	Count	Notes
APIs	20125	Sourcefile not defined for 99109bw_api cannot determine the funcname
Files	5561	folders are completely wrong. e.g. wp-admin, wp-admin\includes repeated over and over
Classes	13914	Appears to be one per method?
Hooks	8459	Post meta empty

Analysis

• It would appear that there's no post_meta data being created.
• The API letters taxonomy logic works though.

[bobbingwide]

The plan of action is to rebuild the complete reference in all sub domains for all components. I’m concerned that the logic that resets the API ref for a component doesn’t clear up all post_meta data and could lead will to further problems. I could either write some draconian SQL that deletes everything for a subdomain or some logic that removes post_meta that refers to non-existent posts.

Before doing that we should validate the newly built API ref for core. AND, to save time, use a smaller component for testing. E.g. Twenty Twenty on its own in develop.wp.a2z

[bobbingwide]

That's more like it. These figures, from core.wp.a2z should be the base for the sitemap pages.

Content type	Count	Notes
APIs	7853
Files	816
Classes	467
Hooks	2416	Hooks tab only shows 2143 - must be 3 others!

There's a problem on the oik-plugins/wordpress page The Classes and Hooks tabs are missing. I had to add some missing code.... but I don't know why.

[bobbingwide]

Running the following SQL I found a whole load of post meta data that hadn't been deleted.

SELECT count(*), meta_key FROM `wp_2_postmeta` 
where post_id not in ( select distinct ID from wp_2_posts ) 
group by meta_key

It's safe to remove them.

delete from wp_2_postmeta where post_id not in ( select distinct ID from wp_2_posts )

SELECT count(*), meta_value FROM `wp_2_postmeta` 
where meta_key = '_oik_hook_plugin' 
group by meta_value

returns 2413

so there must still be 3 'oik_hook' posts which don't have a matching post_meta keyed by '_oik_hook_plugin'

YUP, select * finds them and delete removes them

delete FROM `wp_2_posts` where post_type = 'oik_hook' 
and ID not in ( select post_id from wp_2_postmeta where meta_key = '_oik_hook_plugin' )

[bobbingwide]

On wp.a2z ( local version of wp-a2z.org ) I ran the reset process for all the plugins and themes and ended up with 1 API and 62 hooks left over.

I used wp cli to remove the oik-api post.

wp post delete 15895 --force

and SQL to delete the oik_hooks

delete FROM `wp_posts` where post_type = 'oik_hook' 
and ID not in ( select post_id from wp_postmeta where meta_key = '_oik_hook_plugin' )

and since there's no referential integrity between wp_posts and wp_postmeta

delete from wp_postmeta where post_id not in ( select distinct ID from wp_posts )

[bobbingwide]

I'm now wondering what the API ref would be like if I just build the API ref for WordPress core in core.wp.a2z and the bundled plugins and themes in wp.a2z. Seems like as good a time as any to find out. Might bugger up a few search engine links. Once the full index of ‘core’ is created this can be encapsulated in a WordPress API cache. Links to core functionality can easily be created in the same was as we create links to PHP.

[bobbingwide]

The plan of action is to rebuild the complete reference in all sub domains for all components. I’m concerned that the logic that resets the API ref for a component doesn’t clear up all post_meta data and could lead will to further problems. I could either write some draconian SQL that deletes everything for a subdomain or some logic that removes post_meta that refers to non-existent posts.

I need a method to delete all the post, post_meta and associated taxonomy data from each subdomain where I want to rebuild everything.

[bobbingwide]

I need a method to delete all the post, post_meta and associated taxonomy data from each subdomain where I want to rebuild everything.

that's now delivered in https://github.com/bobbingwide/oik-shortcodes/issues/71

[bobbingwide]

The API reference for each sub domain has now been rebuilt and blog posts written for most sites. there’s a new site map page. The Sites page/menu item should now be the version that gives the live figures from each site.

[bobbingwide]

Closing this now. Moving on to WordPress 5.5... where I appear to have created some more problems for myself on June 15th 2020.