Terrible asset performance

[Stuartemk]

Steps to reproduce the issue

mysqldumpslow -a -s r -t 1 /var/log/mysql/mysql-slow.log

Reading mysql slow query log from /var/log/mysql/mysql-slow.log Count: 621 Time=0.01s (7s) Lock=0.00s (0s) Rows=23347.0 (14498487), server[server]@localhost SELECT id,name,rules,parent_id FROM web_assets WHERE name LIKE 'com_content.%' OR name = 'com_content' OR parent_id = 0

mysqldumpslow -t 1 -s at -r /var/log/mysql/mysql-slow.log

Reading mysql slow query log from /var/log/mysql/mysql-slow.log Count: 594 Time=0.01s (6s) Lock=0.00s (0s) Rows=23347.0 (13868118), server[server]@localhost SELECT id,name,rules,parent_id FROM web_assets WHERE name LIKE 'S' OR name = 'S' OR parent_id = N

Expected result

The results would be indexed or called only when changing, adding or deleting information

Actual result

Server overload on a news page with accumulated daily new articles.

System information (as much as possible)

Setting	Value

  PHP Built On | Linux server 5.3.0-40-generic #32~18.04.1-Ubuntu SMP Mon Feb 3 14:05:59 UTC 2020 x86_64 Database Type | mysql Database Version | 5.7.29-0ubuntu0.18.04.1-log Database Collation | utf8_general_ci Database Connection Collation | utf8mb4_general_ci PHP Version | 7.2.24-0ubuntu0.18.04.3 Web Server | Apache/2.4.29 WebServer to PHP Interface | fpm-fcgi Joomla! Version | Joomla! 3.9.15 Stable [ Amani ] 27-January-2020 15:00 GMT Joomla! Platform Version | Joomla Platform 13.1.0 Stable [ Curiosity ] 24-Apr-2013 00:00 GMT

Additional comments

[alikon]

i'm afraid no indexes can help on a query with LIKE and OR anyway how many articles do you have ? 20K ?

[Stuartemk]

@alikon @wilsonge @SharkyKZ @richard67 @brianteeman @astridx @C-Lodder @Quy @zero24 @zero-24

On a news page with 10 categories such as sports, finance, technology, etc., and with 10 subcategories per category such as soccer, baseball, tennis, etc., adding an article by subcategory and also multilanguage, it is not an exaggeration or crazy to arrive soon to quantities of more than 20,000 items, since the items are also saved and not deleted. Joomla is an excellent and extraordinary CMS for its modular system, user management, flexibility and diversity compared to any other CMS, giving it tremendous power and potential, but one of its few adversities against other CMS such as Drupal or Wp is that since its origin Joomla has a big problem in the structure of the way he handles large volumes of articles and the same thing happens with menus; Neither Drupal nor WP have problems with this handling of large volumes of Articles, specifically the structure of the assets kills all the benefits and benefits of Joomla over other CMS. If it is possible to eliminate the bottleneck of the assets in Joomla, it would no longer have a rival as the best CMS and the same with the bottleneck of large volumes of menus and submenus.

[HLeithner]

Could a fulltext index be used here?

[alikon]

iirc FULLTEXT index can be used starting mysql ≥ 5.6

[HLeithner]

True but would it help j4+?

[Stuartemk]

@HLeithner @alikon @wilsonge @SharkyKZ @richard67 @brianteeman @astridx @C-Lodder @Quy @zero24 @zero-24

Possibly there is ignorance of me in relation to the existence or structure of the Assets, but it definitely has an error in the structure that goes beyond creating indexes, since for example imagine that you have 100,000 word documents on your computer and that for create, change, or delete a word file the computer had to make a comparison with the 100,000 documents ... because that makes joomla with the Assets, it's crazy! It only makes sense for very specific cases in which it is required to do a search for a specific comparative such as finding repeated files and even then joomla does it wrong because it does not define author or any date but does so with 100% of the files that are much worse because what it does with the Assets is also with the plugins, themes, modules etc. since the Assets are also all of them, it is as if you were going to create a word document and the non-only computer will search in the 100,000 Word documents but also include all excel, powerpoint, etc, etc, etc, perhaps only excluding images. It is definitely an error of logic, structure and design.

[ssnobben]

@Stuartemk did you come up with a solution for Joomla for this?

[Fedik]

@Stuartemk for note: spamming everyone will not help to solve the issue at all, even opposite.

Just showing "slow logs" does not helps much, it may be due to bad server performance. Would be much useful if you tell more detail and be more specific, eg: in which cases you have a slow down: when add/edit content, or when doing something else.

All this water about windows and files will no one read.

[Notebit]

I join cause I also have the same issue, a slow query on assets table with a com_content of more than 30K articles, any workaround sofar? That query is executed every time a com_content page is loaded in frontend, with or without cache, and as far as I can see, that query is executed even if I load the page as a visitor or guest

[Notebit]

small update: the assets query takes 22ms in joomla debug but if I run the same query in phpmyadmin it's blazing fast

[Hackwar]

@Notebit 22ms would actually not be slow in my opinion. In any case, here is a PR which should reduce the number of rows in that table, improving performance especially for large sites like these: https://github.com/joomla/joomla-cms/pull/39165 Please test that PR and see if this helps with your issues.