Detect when outgoing http requests (like in the Marketplace) do not work and notify user instead of hanging

[heurteph-ei]

Expected Behavior

Matomo dashboards and configuration pages should be displayed whereas the Marketplace is reacheable or not.

Current Behavior

When Matomo server (on premises) is behind a firewall (and cannot ping www.matomo.org) and the Marketplace component is active, all dashboards, configuration pages and so on are not responding (more than 6 minutes with no response then error mesage about curl problems). In order to solve our problem, we disabled the Marketplace component. But for newbies it is not very simple to understand the issue 😉

Note:

We currently have no CRON configured yet as we discover Matomo on Permises 😉

Possible Solution

Use some multi-threading when doing such minor operations (calling the Marketplace is not the main feature of Matomo 😉)

Your Environment

Mandatory checks

Version PHP >= 7.2.5: ✔ 7.3.20

PDO Extension: ✔

PDO\MYSQL Extension: ✔

MYSQLI Extension: ✔

Extensions requises: ✔ zlib ✔ SPL ✔ iconv ✔ json ✔ mbstring ✔ Reflection

Fonctions requises: ✔ debug_backtrace ✔ eval ✔ hash ✔ gzcompress ✔ gzuncompress ✔ pack

Configuration PHP requise (php.ini): ✔ session.auto_start = 0 ✔ max_execution_time = 0 OR >= 30

Répertoires avec les droits d'accès en écriture: ✔ $DOC_ROOT/tmp ✔ $DOC_ROOT/tmp/assets ✔ $DOC_ROOT/tmp/cache ✔ $DOC_ROOT/tmp/climulti ✔ $DOC_ROOT/tmp/latest ✔ $DOC_ROOT/tmp/logs ✔ $DOC_ROOT/tmp/sessions ✔ $DOC_ROOT/tmp/tcpdf ✔ $DOC_ROOT/tmp/templates_c

Répertoires avec accès en écriture pour Gestionnaire de balises: ✔ $DOC_ROOT/js

Optional checks

Intégrité du fichier: ✔

Statut du traqueur: ✔

Limite mémoire: ✔ 128M

Fuseau horaire: ✔

Ouvrez l'URL: ✔ curl

PageSpeed désactivé: ✔

GD > 2.x + Freetype (graphiques): ✔

Autres extensions: ✔ json ✔ libxml ✔ dom ✔ SimpleXML

Autres fonctions: ✔ shell_exec ✔ set_time_limit ✔ mail ✔ parse_ini_file ✔ glob ✔ gzopen ✔ md5_file

Système de fichiers: ✔

Setup Cron (faster report loading): ⚠ Warning: Pour des performances optimales et un Matomo rapide, il est vivement recommandé de mettre en place une entré CRON pour archiver automatiquement vos rapports, et de désactiver le déclanchement depuis l'interface depuis les paramètres Matomo.

Setup Cron - Gestion des processus via la ligne de commande: ✔ Ok

Capacités de la base de données: ✔ UTF8mb4 charset ⚠ Warning: LOAD DATA INFILE
L'utilisation de LOAD DATA INFILE va grandement accélérer la vitesse du processus d'archivage de Matomo. Pour le rendre disponible à Matomo, essayez de mettre à jour PHP & MySQL et assurez-vous que votre utilisateur MySQL a les privilèges de FILE.
Si votre serveur Matomo suit des sites à fort trafic (ex. > 100000 pages par mois), nous recommandons d'essayer de corriger ce problème.
Erreur: LOAD DATA INFILE failed... Error was:
Try #1: LOAD DATA INFILE : SQLSTATE[28000]: Invalid authorization specification: 1045 Access denied for user (using password: YES)[28000],
Try #2: LOAD DATA LOCAL INFILE : SQLSTATE[42000]: Syntax error or access violation: 1148 The used command is not allowed with this MySQL version[42000]
✔ CREATE TEMPORARY TABLES ✔ Changing transaction isolation level

Taille maximale des paquets: ⚠ Warning: Il est important de configurer une taille 'max_allowed_packet' dans votre base de données MySQL d'au moins 64MB. 32MB configuré en ce moment.

Connexion SSL forcée: ⚠ Warning: Nous recommandons d'utiliser Matomo uniquement avec des connexions sécurisées SSL. Pour empêcher l'accès non sécurisée par HTTP, ajoutez force_ssl = 1 à la section General dans votre fichier conf/config.ini.php de Matomo.

Géolocalisation: ⚠ Warning: La géolocalisation fonctionne, mais vous n'utilisez pas un des fournisseurs recommandés. Si vous devez importer des fichiers de journaux ou bien effectuer quoi que ce soit d'autre qui requiert la définition des adresses IP, utilisez l'implémentation PHP GeoIP l'extension maxminddb.

Mettre à jour via HTTPS: ✔

Traceur JavaScript inscriptible ("/matomo.js"): ✔

Heatmap & Session Recording Tracking: ⚠ Warning: Requesting '$MATOMO_URLplugins/HeatmapSessionRecording/configs.php?idsite=1&trackerid=5lX6EM&url=http%3A%2F%2Ftest.test%2F' resulted in an SSL error. Maybe you are using a self signed certificate? Please open the URL manually in a browser to see if the response contains 'Piwik.HeatmapSessionRecording'. If not, you might need to modify your server configuration as this file needs to be accessible via a browser from the Internet or Intranet.

Informational results

Matomo Version: 4.0.5

Matomo Update History: 4.0.5,

Matomo Install Version: 4.0.5

PHP_OS: Linux

PHP SAPI: apache2handler

Timezone Version: 0.system

PHP Timezone: UTC

PHP Time: 1614954151

PHP Datetime: 2021-03-05 14:22:31

PHP INI max_execution_time: 30

PHP INI post_max_size: 8M

PHP INI max_input_vars: 1000

PHP INI zlib.output_compression:

Curl Version: 7.29.0, NSS/3.44

Suhosin Installed: 0

DB Prefix: matomo_

DB Charset: utf8mb4

DB Adapter: PDO\MYSQL

MySQL Version: 5.7.33-enterprise-commercial-advanced-log

Num Tables: 69

Browser Segment Archiving Enabled: 1

Development Mode Enabled: 0

Internet Enabled: 1

Multi Server Environment: 0

Custom User Path: 0

Custom Include Path: 0

Plugins Activated: API, AbTesting 4.0.6, Actions, ActivityLog 4.0.2, Annotations, BulkTracking, Cohorts 4.0.2, Contents, CoreAdminHome, CoreConsole, CoreHome, CorePluginsAdmin, CoreUpdater, CoreVisualizations, CustomDimensions, CustomJsTracker, CustomReports 4.0.4, Dashboard, DevicePlugins, DevicesDetection, Diagnostics, Events, Feedback, FormAnalytics 4.0.2, Funnels 4.0.3, GeoIp2, Goals, Heartbeat, HeatmapSessionRecording 4.0.6, ImageGraph, Insights, Installation, Intl, IntranetMeasurable, LanguagesManager, Live, LoginLdap 4.3.0, MobileMessaging, Monolog, Morpheus, MultiSites, Overlay, PagePerformance, PrivacyManager, ProfessionalServices, Proxy, Referrers, Resolution, RollUpReporting 4.0.1, RssWidget, SEO, ScheduledReports, SearchEngineKeywordsPerformance 4.1.1, SegmentEditor, SitesManager, TagManager, Tour, Transitions, TwoFactorAuth, UserCountry, UserCountryMap, UserId, UserLanguage, UsersFlow 4.0.2, UsersManager, VisitFrequency, VisitTime, VisitorInterest, VisitsSummary, WebsiteMeasurable, Widgetize

Plugins Deactivated: CustomVariables, DBStats, Ecommerce, Login, LoginSaml 4.0.3, Marketplace (yes we had to deactivate it!), MediaAnalytics 4.0.2, MobileAppMeasurable, MultiChannelConversionAttribution 4.0.2, Provider, WhiteLabel 4.0.1, WooCommerceAnalytics 4.0.3

Plugins Invalid:

Server Info: Apache

Had visits in last 1 day: 1

Had visits in last 3 days: 1

Had visits in last 5 days: 1

Archive Time Last Started: 0

Archive Time Last Finished: 0

Num invalidations: 80 queued, 0 in progress

User Agent: Mozilla/5.0 (Windows NT 6.3; Win64; x64; rv:78.0) Gecko/20100101 Firefox/78.0

Browser Language: fr,fr-fr,en-us,en

Anonymize Referrer:

Do Not Track enabled: 0

[Findus23]

Hi,

Maybe this guide might help you: https://matomo.org/faq/troubleshooting/faq_16646/

[heurteph-ei]

I agree, BUT as by default, the configuration is with internet enabled and marketplace active, another newbie like me will encounter the same disapointing issue... Matomo should not freeze during 5 minutes for such a minor feature.

[diosmosis]

Hi @heurteph-ei, you're right, Matomo should behave better in this situation, letting the user know there needs to be a configuration change.

[tsteur]

BTW I'm thinking that it might be a lot slower than usual because premium features are installed. Without this, Matomo core would likely load a lot faster when internet is disabled. Not saying nothing needs to be done just adding this as an FYI.

[tsteur]

Maybe a system check would help to detect if internet is enabled and then point out the setting may need to be disabled?

[diosmosis]

That would help, but only if the system check was reachable, which it might not be in this case.