Develop charts for Daily Trace Summary files

bobbingwide commented 3 years ago

A while ago, the logic written in vt.php was copied and cobbled into wp-top12. It was extended as vt-stats.php and vt-top12.php. Both routines produce CSV files summarising daily trace output in a variety of ways. See the readme file for https://github.com/bobbingwide/wp-top12

I now want to convert the batch routines into Admin pages that extend oik-bwtrace. and provide a variety of charts to be displayed in the Dashboard. The admin pages will allow the user to select the required analysis.

Requirement

Ability to analyse the Daily Trace Summary reports in the WordPress dashboard
Choose the file or files to be analysed.
Choose from a range of reports

Proposed solution

[x] Add an admin page
[ ] Support WordPress Multi Site summary trace files
[x] Use/extend code from files in wp-top12 and pompey-chart
[x] Eventually the code will have to be converted to shared libraries.

The admin page will display:

[x] A form for input file and chart selection.
[x] Selected chart
[x] The chart's raw data in tabular form.

Form fields

Name	Selection fields	Values
file	File to analyse	List of daily trace summary files
report	Report	See Reports below
type	Chart type	Line, Bar, Pie
display	Display values	Count, Elapsed, Average, Percentage count, Percentage elapsed, Accumulated count percentage, Accumulated elapsed percentage
having	Having	Select groups having greater or equal to the given number of items

Reports

Report	Displays
Request types	Requests classified by type
Stripped Request URIs	Grouped by the Uniform Resource Identifier
Stripped Request URIs Top Level	Grouped by the first part of the Uniform Resource Identifier
Elapsed	Grouped by Elapsed execution time ( in twentieth's of a second ).
Hook counts	Grouped by Hook counts
Remote IP	Grouped by IP address
Hourly	Grouped by time in the day

bobbingwide commented 3 years ago

I'm going to transfer this to wp-top12 until the common code's been transferred back to slog, or shared libraries.

bobbingwide commented 3 years ago

It started as issue 5 and now, after transferring to and from wp-top12 it's issue 6.

bobbingwide commented 3 years ago

There's a problem when the orginal record in the daily trace summary file has more commas than expected. I detected this when producing the Stripped request URIs report. The chart displayed a load of undefined labels.

Where encountered: https://s.b/oikcom/wp-admin/options-general.php?page=slog
Source file: C:/apache/htdocs/wordpress/wp-content/plugins/slog-bloat/working/2021/oikcouk0126.csv

Explanation: Two unexpected WP-CLI commands

"/usr/local/cpanel/3rdparty/wp-toolkit/plib/vendor/wp-cli/wp-cli/php/boot-fs.php --path=/home/oikcouk/public_html --no-color config-settings get --params=DB_CHARSET,DB_NAME,DB_USER,DB_PASSWORD,DB_HOST,DOMAIN_CURRENT_SITE,PATH_CURRENT_SITE,MULTISITE,SUBDOMAIN_INSTALL,WP_AUTO_UPDATE_CORE,WPCACHEHOME,WP_DEBUG,WP_DEBUG_LOG,WP_DEBUG_DISPLAY,SCRIPT_DEBUG,SAVEQUERIES,WP_AUTO_UPDATE_CORE,WP_HOME,WP_SITEURL,CONCATENATE_SCRIPTS,DISALLOW_FILE_EDIT,WP_CACHE_KEY_SALT,DISABLE_WP_CRON --format=json",,0.200017,7.3.26,1748,3182,466,1,441,51,10,5,2,0.0047807693481445,,,,5044,127.0.0.1,0.199963,2021-01-26T11:48:46+00:00,,GET
/xmlrpc.php,,0.412201,7.3.26,1736,5396,595,50,832,58,36,20,10,0.0083532333374023,,,,13857,193.142.146.202,0.412134,2021-01-26T11:48:46+00:00,Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML; like Gecko) Chrome/56.0.2924.90 Safari/537.36 2345Explorer/9.3.2.17331,POST
"/usr/local/cpanel/3rdparty/wp-toolkit/plib/vendor/wp-cli/wp-cli/php/boot-fs.php --path=/home/oikcouk/public_html --no-color config-settings get --params=DB_CHARSET,DB_NAME,DB_USER,DB_PASSWORD,DB_HOST,DOMAIN_CURRENT_SITE,PATH_CURRENT_SITE,MULTISITE,SUBDOMAIN_INSTALL,WP_AUTO_UPDATE_CORE,WPCACHEHOME,WP_DEBUG,WP_DEBUG_LOG,WP_DEBUG_DISPLAY,SCRIPT_DEBUG,SAVEQUERIES,WP_AUTO_UPDATE_CORE,WP_HOME,WP_SITEURL,CONCATENATE_SCRIPTS,DISALLOW_FILE_EDIT,WP_CACHE_KEY_SALT,DISABLE_WP_CRON --format=json",,0.202267,7.3.26,1748,3182,466,1,441,51,10,5,2,0.0046060085296631,,,,5044,127.0.0.1,0.202223,2021-01-26T11:48:47+00:00,,GET

Side issues

What's TSOhost doing running these commands anyway?
How to block the thousands of xmlrpc.php requests noted that day. All from the same IP address.

bobbingwide commented 3 years ago

How to block the thousands of xmlrpc.php requests noted that day. All from the same IP address.

Update your .htaccess file, adding this:

# Block WordPress xmlrpc.php requests
<Files xmlrpc.php>
<IfModule mod_authz_core.c>
    Require all denied
</IfModule>
<IfModule !mod_authz_core.c>
    Order deny,allow
    Deny from all
</IfModule>
</Files>

This is safer than trying to use Wordfence's Firewall to Immediately block IPs that access these URLs since there's a gotcha. From their website https://www.wordfence.com/help/firewall/options/#immediately-block-urls

We only recommend this feature if you are trying to catch a specific hacker to block them, or if you are trying to catch hackers that are trying to exploit a known vulnerability or page on your site. Be careful not to visit the banned URL yourself by accident, as you will be blocked instantly.

bobbingwide commented 3 years ago

Requirement

Easy selection of the daily trace summary file to analyse

Proposed solution

Use a selection list populating the files from the trace files directory.
Sort by date descending.

For slog-bloat, if we're going to be running comparisons between sites, we need to be able to download the files from the server and put them into a working directory.

This may need to be done with a download button. See Andrew's code for forcing downloads?

bobbingwide / slog