search

matomo-org / matomo-log-analytics

Import any kind of server logs in Matomo for powerful log analytics. Universal log file parsing and reporting.
https://matomo.org/log-analytics/
GNU General Public License v3.0
225 stars 118 forks source link

Importer very, very slow after upgrade to Matomo 4 #294

Open L3on1d opened 3 years ago

L3on1d commented 3 years ago

Hi!

Importer very slow after upgrade to Matomo 4.

Before upgrade my RPS was: 1052.24 requests per second, now after upgrade to 4.0.5, I'm getting 67.74 requests per second :( Any thoughts?

Matomo version: 4.0.5 MySQL version: 10.4.17 (DB tables converted to utf8mb4) PHP version: 7.3.24 Python version: 3.9

sgiehl commented 3 years ago

might that be the same reason as for https://github.com/matomo-org/matomo/issues/16924 ?

L3on1d commented 3 years ago

@sgiehl thank you, looks like pretty similar, I have updated Model.php, but speed still awful.

image

L3on1d commented 3 years ago

Sometimes this error appearing during the import:

PHP message: Error in Matomo (tracker): Error query: SQLSTATE[40001]: Serialization failure: 1213 Deadlock found when trying to get lock; try restarting transaction In query: UPDATE mtm_log_visit SET profilable = ?, visitor_seconds_since_order = ?, visit_exit_idaction_url = ?, visit_total_actions = visit_total_actions + 1 , visit_total_interactions = visit_total_interactions + 1 , visit_total_time = ? WHERE idsite = ? AND idvisit = ? Parameters: array ( 0 => 0, 1 => NULL, 2 => 2588680546, 3 => 589, 4 => 6, 5 => '3037957010', )" while reading response header from upstream