matomo-org / matomo

Empowering People Ethically with the leading open source alternative to Google Analytics that gives you full control over your data. Matomo lets you easily collect data from websites & apps and visualise this data and extract insights. Privacy is built-in. Liberating Web Analytics. Star us on Github? +1. And we love Pull Requests!
https://matomo.org/
GNU General Public License v3.0
19.91k stars 2.65k forks source link

Error array_combine(): Argument #1 ($keys) and argument #2 ($values) must have the same number of elements in PivotByDimension #19806

Open tsteur opened 2 years ago

tsteur commented 2 years ago

URL: /index.php?module=API&format=CSV&idSite=1&period=week&date=2022-10-02&method=MarketingCampaignsReporting.getName&filter_update_columns_when_show_all_goals=-1&filter_show_goal_columns_process_goals=ecommerceOrder,2,3&expanded=1&format_metrics=1&pivotBy=MarketingCampaignsReporting.CombinedKeywordContent&pivotByColumnLimit=20&translateColumnNames=1&language=en&token_auth=XYZANONYMIZED&force_api_session=1&filter_limit=5

Referrer: /index.php?module=CoreHome&action=index&idSite=1&period=day&date=yesterday

GET: {"module":"API","format":"CSV","idSite":"1","period":"week","date":"2022-10-02","method":"MarketingCampaignsReporting.getName","filter_update_columns_when_show_all_goals":"-1","filter_show_goal_columns_process_goals":"ecommerceOrder,2,3","expanded":"1","format_metrics":"1","pivotBy":"MarketingCampaignsReporting.CombinedKeywordContent","pivotByColumnLimit":"20","translateColumnNames":"1","language":"en","token_auth":"XYZANONYMIZED","force_api_session":"1","filter_limit":"5"}

stack trace:

Error: {"message":"array_combine(): Argument #1 ($keys) and argument #2 ($values) must have the same number of elements","file":"core\/DataTable\/Filter\/PivotByDimension.php","line":254,"request_id":"ec5c3","backtrace":" on core\/DataTable\/Filter\/PivotByDimension.php(254)\n#0 core\/DataTable\/Filter\/PivotByDimension.php(254): array_combine(Array, Array)\n#1 [internal function]: Piwik\DataTable\Filter\PivotByDimension->Piwik\DataTable\Filter\{closure}(Object(Piwik\DataTable))\n#2 core\/DataTable.php(511): call_user_func_array(Object(Closure), Array)\n#3 core\/DataTable.php(617): Piwik\DataTable->filter(Object(Closure), Array)\n#4 core\/API\/DataTablePostProcessor.php(304): Piwik\DataTable->applyQueuedFilters()\n#5 core\/API\/DataTablePostProcessor.php(134): Piwik\API\DataTablePostProcessor->applyQueuedFilters(Object(Piwik\DataTable))\n#6 core\/API\/ResponseBuilder.php(190): Piwik\API\DataTablePostProcessor->process(Object(Piwik\DataTable))\n#7 core\/API\/ResponseBuilder.php(104): Piwik\API\ResponseBuilder->handleDataTable(Object(Piwik\DataTable))\n#8 core\/API\/Request.php(272): Piwik\API\ResponseBuilder->getResponse(Object(Piwik\DataTable), 'MarketingCampai...', 'getName')\n#9 core\/Context.php(28): Piwik\API\Request->Piwik\API\{closure}()\n#10 core\/API\/Request.php(273): Piwik\Context::executeWithQueryParameters(Array, Object(Closure))\n#11 plugins\/API\/Controller.php(45): Piwik\API\Request->process()\n#12 [internal function]: Piwik\Plugins\API\Controller->index()\n#13 core\/FrontController.php(631): call_user_func_array(Array, Array)\n#14 core\/FrontController.php(169): Piwik\FrontController->doDispatch('API', false, Array)\n#15 core\/dispatch.php(32): Piwik\FrontController->dispatch()\n#16 index.php(25): require_once('c...')\n#17 {main}","safemode_backtrace":"#0 [internal function]: Piwik\Plugins\Cloud\Controller->safemode(Array)\n#1 \/core\/FrontController.php(631): call_user_func_array(Array, Array)\n#2 \/core\/FrontController.php(169): Piwik\FrontController->doDispatch('Cloud', 'safemode', Array)\n#3 \/core\/FrontController.php(100): Piwik\FrontController->dispatch('CorePluginsAdmi...', 'safemode', Array)\n#4 \/core\/FrontController.php(140): Piwik\FrontController::(Array)\n#5 \/core\/FrontController.php(190): Piwik\FrontController::(Object(ValueError))\n#6 \/core\/dispatch.php(32): Piwik\FrontController->dispatch()\n#7 \/index.php(25): require_once('\/c...')\n#8 {main}"}

tsteur commented 2 years ago

This happened again

tsteur commented 2 years ago

This happened again.

tsteur commented 1 year ago

This happened again today. URL: ?module=API&format=TSV&idSite=1&period=week&date=2022-11-28&method=MarketingCampaignsReporting.getSourceMedium&filter_update_columns_when_show_all_goals=-1&filter_show_goal_columns_process_goals=1,2,3,4,5&flat=1&include_aggregate_rows=1&pivotBy=MarketingCampaignsReporting.CampaignName&pivotByColumnLimit=20&translateColumnNames=1&language=fr&segment=campaignName%3D%40ffff&filter_limit=100

d--j commented 1 year ago

Happened here too

$this->fullVersionList is

array ( 
  0 =>    array (     '' => 'Not_A ',     'version' => '99.0.0.0',   ),  
  1 =>    array (     '' => 'Google Chrome',     'version' => '109.0.5414.117',   ), 
  2 =>    array (     '' => 'Chromium',     'version' => '109.0.5414.117',   ), 
)

so there is no brand array element (it's '' in this case).

This is the backtrace when it happended:

Uncaught exception in /var/www/vendor/matomo/device-detector/ClientHints.php line 193:
array_combine(): Argument #1 ($keys) and argument #2 ($values) must have the same number of elements
#0 /var/www/vendor/matomo/device-detector/ClientHints.php(193): array_combine()
#1 /var/www/core/DeviceDetector/DeviceDetectorFactory.php(48): DeviceDetector\ClientHints->getBrandList()
#2 /var/www/core/DeviceDetector/DeviceDetectorFactory.php(30): Piwik\DeviceDetector\DeviceDetectorFactory::getNormalizedUserAgent()
#3 /var/www/core/Tracker/VisitExcluded.php(198): Piwik\DeviceDetector\DeviceDetectorFactory->makeInstance()
#4 /var/www/core/Tracker/VisitExcluded.php(72): Piwik\Tracker\VisitExcluded->isNonHumanBot()
#5 /var/www/plugins/CoreHome/Tracker/VisitRequestProcessor.php(99): Piwik\Tracker\VisitExcluded->isExcluded()
#6 /var/www/core/Tracker/Visit.php(164): Piwik\Plugins\CoreHome\Tracker\VisitRequestProcessor->processRequestParams()
#7 /var/www/core/Tracker.php(172): Piwik\Tracker\Visit->handle()
#8 /var/www/plugins/QueuedTracking/Queue/Processor/Handler.php(46): Piwik\Tracker->trackRequest()
#9 /var/www/plugins/QueuedTracking/Queue/Processor.php(194): Piwik\Plugins\QueuedTracking\Queue\Processor\Handler->process()
#10 /var/www/plugins/QueuedTracking/Queue/Processor.php(143): Piwik\Plugins\QueuedTracking\Queue\Processor->processRequestSets()
#11 /var/www/plugins/QueuedTracking/Commands/Process.php(86): Piwik\Plugins\QueuedTracking\Queue\Processor->process()
#12 /var/www/vendor/symfony/console/Symfony/Component/Console/Command/Command.php(257): Piwik\Plugins\QueuedTracking\Commands\Process->execute()
#13 /var/www/vendor/symfony/console/Symfony/Component/Console/Application.php(874): Symfony\Component\Console\Command\Command->run()
#14 /var/www/vendor/symfony/console/Symfony/Component/Console/Application.php(195): Symfony\Component\Console\Application->doRunCommand()
#15 /var/www/core/Console.php(108): Symfony\Component\Console\Application->doRun()
#16 [internal function]: Piwik\Console->originDoRun()
#17 /var/www/core/Console.php(147): call_user_func()
#18 /var/www/core/Access.php(670): Piwik\Console->Piwik\{closure}()
#19 /var/www/core/Console.php(148): Piwik\Access::doAsSuperUser()
#20 /var/www/core/Console.php(87): Piwik\Console->doRunImpl()
#21 /var/www/vendor/symfony/console/Symfony/Component/Console/Application.php(126): Piwik\Console->doRun()
#22 /var/www/console(32): Symfony\Component\Console\Application->run()
#23 {main}

I just patched getBrandList

https://github.com/matomo-org/device-detector/blob/c12bcb63458f8cf2a058049ce5a0960b211ec208/ClientHints.php#L188-L198

to also handle this input.

    public function getBrandList(): array
    {
        if (\count($this->fullVersionList)) {
            if (is_array(current($this->fullVersionList)) && array_key_exists('brand', current($this->fullVersionList))) {
                return \array_combine(
                    \array_column($this->fullVersionList, 'brand'),
                    \array_column($this->fullVersionList, 'version')
                ) ?: [];
            } else {
                return @\array_combine(
                    @\array_column($this->fullVersionList, ''),
                    @\array_column($this->fullVersionList, 'version')
                ) ?: [];
            }
        }

        return [];
    }

That's of course not the right fix – that would be to figure out why there is no brand key in this instance.

sgiehl commented 1 year ago

@d--j the error your are having seems to be fully unrelated to the initial issue here. Even though the error message is the same. It looks like you are using queued tracking. If I remember correctly there had been other reports about incorrectly transferred user agent data. It might though be a problem with that plugin maybe. Maybe check the plugins repo for that: https://github.com/matomo-org/plugin-QueuedTracking/issues. I'll nevertheless will create a patch for device detector, so it won't throw a warning/deprecation anymore, but simply discard such incorrect values.

samjf commented 1 year ago

I got a bug like this with the same stack trace and it has the following params:

{
    "module": "API",
    "format": "HTML",
    "idSite": "1",
    "period": "range",
    "date": "2023-06-01,2023-06-02",
    "method": "MarketingCampaignsReporting.getSourceMedium",
    "filter_update_columns_when_show_all_goals": "0",
    "filter_show_goal_columns_process_goals": "ecommerceOrder,1,2,3,4,5,6",
    "expanded": "1",
    "pivotBy": "MarketingCampaignsReporting.CampaignName",
    "pivotByColumnLimit": "20",
    "idGoal": "0",
    "token_auth": "XYZANONYMIZED",
    "force_api_session": "1",
    "filter_limit": "10"
}
tsteur commented 1 year ago

This happened again.

GET: {"module":"API","format":"TSV","idSite":"1","period":"range","date":"2023-03-01,2023-05-31","method":"MarketingCampaignsReporting.getSourceMedium","filter_update_columns_when_show_all_goals":"-1","filter_show_goal_columns_process_goals":"ecommerceOrder,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56","expanded":"1","pivotBy":"MarketingCampaignsReporting.CampaignName","pivotByColumnLimit":"20","translateColumnNames":"1","language":"en","token_auth":"XYZANONYMIZED","force_api_session":"1","filter_limit":"-1"}

achangjvi22 commented 1 year ago

I get the same again yesterday and a few times over the last few months - despite the update with 4.14.1

However the error that jams-up the queue is caused by ClientHints.php as in the log below.

Uncaught exception in ..../vendor/matomo/device-detector/ClientHints.php line 193: array_combine(): Argument #1 ($keys) and argument #2 ($values) must have the same number of elements

Since we have so much traffic I'm unable to ascertain what the actual entry is.. all i can do to get it working again is to delete the item or items at the front of the queue. and process manually.

Is it the Queued Tracking Plugin that is causing the issue or does it have to do with something in the device-detector? Either way is causing our queues to sieze and buildup endlessly

sgiehl commented 1 year ago

@achangjvi22 You particular error is unrelated to the one reported in this issue. Anyway, the error you described should have been fixed with Matomo 4.14.0. You could maybe check how the code in /vendor/matomo/device-detector/ClientHints.php looks around line 193. There should be checks, that the count matches exactly.

tsteur commented 1 year ago

This issue happened for us again

tsteur commented 10 months ago

This issue is still happening from time to time.

This fatal error happened on 5.0.0-rc9

Error: {"message":"array_combine(): Argument #1 ($keys) and argument #2 ($values) must have the same number of elements","file":"\/core\/DataTable\/Filter\/PivotByDimension.php","line":254,"request_id":"43ff0","backtrace":" on \/core\/DataTable\/Filter\/PivotByDimension.php(254)\n#0 \/core\/DataTable\/Filter\/PivotByDimension.php(254): array_combine(Array, Array)\n#1 [internal function]: Piwik\DataTable\Filter\PivotByDimension->Piwik\DataTable\Filter\{closure}(Object(Piwik\DataTable))\n#2 \/core\/DataTable.php(513): call_user_func_array(Object(Closure), Array)\n#3 \/core\/DataTable.php(635): Piwik\DataTable->filter(Object(Closure), Array)\n#4 \/core\/API\/DataTablePostProcessor.php(316): Piwik\DataTable->applyQueuedFilters()\n#5 \/core\/API\/DataTablePostProcessor.php(134): Piwik\API\DataTablePostProcessor->applyQueuedFilters(Object(Piwik\DataTable))\n#6 \/core\/API\/ResponseBuilder.php(190): Piwik\API\DataTablePostProcessor->process(Object(Piwik\DataTable))\n#7 \/core\/API\/ResponseBuilder.php(104): Piwik\API\ResponseBuilder->handleDataTable(Object(Piwik\DataTable))\n#8 \/core\/API\/Request.php(278): Piwik\API\ResponseBuilder->getResponse(Object(Piwik\DataTable), 'MarketingCampai...', 'getSourceMedium')\n#9 \/core\/Context.php(28): Piwik\API\Request->Piwik\API\{closure}()\n#10 \/core\/API\/Request.php(279): Piwik\Context::executeWithQueryParameters(Array, Object(Closure))\n#11 \/plugins\/API\/Controller.php(45): Piwik\API\Request->process()\n#12 [internal function]: Piwik\Plugins\API\Controller->index()\n#13 \/core\/FrontController.php(637): call_user_func_array(Array, Array)\n#14 \/core\/FrontController.php(169): Piwik\FrontController->doDispatch('API', false, Array)\n#15 \/core\/dispatch.php(32): Piwik\FrontController->dispatch()\n#16 \/index.php(25)

URL: /index.php?module=API&format=CSV&idSite=29&period=range&date=2023-03-08,2023-12-22&method=MarketingCampaignsReporting.getSourceMedium&filter_update_columns_when_show_all_goals=0&filter_show_goal_columns_process_goals=1,2,3,4,5&expanded=1&pivotBy=MarketingCampaignsReporting.CampaignName&pivotByColumnLimit=20&translateColumnNames=1&language=fr&idGoal=0&token_auth=XYZANONYMIZED&force_api_session=1&filter_limit=10

Referrer: /index.php?module=CoreHome&action=index&idSite=29&period=day&date=yesterday&updated=1